�ݺ�ߣ

Tying Flies, Catching Fish | Aug 2022 | 1
Who am I?
Old man yells at the Cloud (engineers)
@behemphi
● Former DBA (MySQL, Postgres, Oracle)
● Former Developer (PHP, Python, Java)
● Former Evangelist (Stackengine Jesus!)
● Former Director of Operations, Infra, Devops,
Cloud Engineering
● Current Engineering Manager
● Community contributor (tho not lately)
Tackle is the onramp to cloud marketplaces. Those
are like app stores for the cloud. If you aren’t selling
there you should ask me about it. I was amazed by
the market opportunities!

Debt is a Tool
Takes one to know one and I am, in fact, a tool
(ask Laura, she’ll tell you)

Consumer Debt - Can be good
Real examples from my personal ﬁnance
I recently borrowed $30k for a car b/c the
cost of credit was 2%. I’ll make more on
investments, so I used debt as a tool.
Car - Depreciating Asset
Borrowed $160k in 2004. Owe about $70k
today. Home worth $540k. Debt allowed me
to purchase this asset.
Home #1 - Appreciating Asset
Pay it oﬀ every month. Costs built into the
price of goods, so points are a way to recoup
this.
Credit Card - Short term debt
today. Home appreciated 15%. Debt allowed
purchase of dream much cheaper.
Home #2 - By in to market

Consumer Debt - Can be good
Real examples from my personal ﬁnance
I recently borrowed $30k for a car b/c the
cost of credit was 2%. I’ll make more on
investments, so t
Car - Depreciating Asset
today. Home worth $540k. Debt allowed me
to purchase this asset.
Home #1 - Appreciating Asset
Pay it oﬀ every month. Costs built into the
price of goods, so points are a way to recoup
this.
Credit Card - Short term debt
today. Home appreciated 15%. Debt allowed
purchase of dream much cheaper.
Home #2 - Market Opportunity

Technical Debt is a Tool in for Business

Metrics of Debt
Cost of
Credit
Unlike consumer debt,
we don’t have solid ways
to measure what we are
getting ourselves into …
We also don’t have
scoring to tell us if we
can aﬀord more debt.
In simple terms, if I
borrow 100 tests not
written today and that
saves me 10 days of
work now …
what are the costs in
time one year from
now?
You got one with
every loan you ever
had.
Is there an origination
fee? Late payment
penalty? Fixed interest
or variable? Balloon
payment?
What is gained by
taking on the debt?
● Immediate
term?
● Long term?
Term
sheet
ROI

Business scenario
Business opportunity
Choosing Debt
A major reseller says if we can oﬀer a speciﬁc
capability they will enter into a $4m biz
agreement with us.
We have to get this capability to market in 3
months due to certain marketing and revenue
cycles with the reseller.
We decide to do 20% (rather than 60%) test
coverage while using dead easy tech (MySQL)
rather than what we anticipate to be more
appropriate tech (Kafka + Elasticsearch)
Term Sheet

Lesson #1
Understand Cost of Credit
● We must quantify the cost of the debt in clear, concise terms or
our leadership cannot be blamed for treating this action as
“free.”
● What if we could show our $4m opportunity will cost $2.5m over
two years? Maybe it’s not the best opportunity?

Business scenario
Payment Schedule
Choosing Debt
We decided to do 20% (rather than 60%) test
coverage
Assume that to get to 60% on initial development
it was 1000 hours of work.
● Assume it’s 1500 hours after the fact.
● Assume 2 Sev1 and 4 Sev2 incidents
resulting from lack of coverage.
Can getting to 60% coverage (or better) in three
months while keeping up with current
maintenance burden and new project work?
Can we make the payments?

Some numbers
Time
We bitch mercilessly
about how expensive
incidents are …
How expensive are they
exactly?
We know an engineer
costs $120/hour fully
loaded.
So the cost of credit
on extra dev time is
500 hrs * $120 =
$60,000
We now the cost of a
Sev1 incident is roughly
$20,000 most of that is
the 100 hours of time
lost
Sev2 is roughly the
same. Again most of that
is the 100 hours lost.
6 incidents@$20k each =
$120,000
What is next on the
roadmap that will be
delayed by the 1100
hours in time and toil?
Is there an upcoming
hard deadline on any of
it?
1100 hrs over three
months is roughly 4 full
time engineers during
that time.
Toil Op Cost

Some numbers
Time
We bitch mercilessly
about how expensive
incidents are …
How expensive are they
exactly?
We know an engineer
costs $120/hour fully
loaded.
So the cost of credit
on extra dev time is
500 hrs * $120 =
$60,000
We now the cost of a
Sev1 incident is roughly
$20,000 most of that is
the 100 hours of time
lost
Sev2 is roughly the
same. Again most of that
is the 100 hours lost.
6 incidents@$20k each =
$120,000
What is next on the
delayed by the 1100
hours in time and toil?
Is there an upcoming
hard deadline on any of
it?
1100 hrs over three
months is roughly 4 full
time engineers during
that time.
Toil Op Cost

Lesson #2
Team Credit Score
● We must have heuristics that allow us to understand if we can
make payments
● Remember this is _in addition_ to the cost to develop the hack
and operate it.

Business scenario
Balloon Payment
Choosing Debt
We decided to use dead easy tech (MySQL)
rather than what we anticipate to be more
appropriate tech (Kafka + Elasticsearch).
We got the $4m, but our system is not able
to scale to reach the $200m of addressable
ARR.
Balloon Payment?

ROI
Time
For the price of
development and
manageable debt we
earned $4m and learned
$200m is available.
Good debt?
Not yet. Can we get the
$200m reasonably?
It will take us 4 engineers
working 6 months to
make the system
scalable, move the data,
test and deploy to the
new Kafka +
Elasticsearch.
That is 4000 hours at
$120k =
$480,000
Because service level has
dropped we will lose
$500,000 of our
customers.
We will incur another 12
major incidents at 100
hours each.
$120k + $500k
$644,000
What is next on the
delayed by the 4000
hours of opportunity
cost?
Likely it is NOT a
proven $200m ARR
opportunity.
Opportunity cost = $0
Toil Op Cost

ROI
Time
For the price of
development and
manageable debt we
earned $4m and learned
$200m is available.
Good debt?
Not yet. Can we get the
$200m reasonably?
It will take us 4 engineers
working 6 months to
make the system
scalable, move the data,
test and deploy to the
new Kafka +
Elasticsearch.
$120k =
$480,000
Because service level has
dropped we will lose
$500,000 of our
customers.
We will incur another 12
major incidents at 100
hours each.
$120k + $500k
$644,000
What is next on the
delayed by the 4000
hours of opportunity
cost?
Likely it is NOT a
proven $200m ARR
opportunity.
Opportunity cost = $0
Toil Op Cost

Lesson #3
Understand Balloon Payments
● Given this works, we are spending and extra $1.2m ($1.4 total
from ﬁrst round) on debt to achieve a potential $200m ARR.
● Rework isn’t really all that expensive in the context of what we
learned and the opportunity available.

So why do we bitch about TechDebt all the
time?
Seriously the term has lost its meaning as just that thing everyone complains about
and does nothing about.

We Fail to pay
$free.99
Clearly we are not
Lannisters.
Because we don’t take
care to understand a
given debt, leadership
thinks it is “free.”
This is _our_ fault. We
need to make debt
more easy to
understand. Esp since
the metaphor is so
easy to understand.
Because we don’t have a
payment schedule for
each debt we take on,
we cannot say how
much of our velocity is
just making payments.
So we look slower and
slower to the biz.
We do not demand
from product the
revenue side in the
same way the expect
velocity and other
metrics from us.
Accum P = R - C

Lesson #4
We fail to pay because we don’t have a strategy for pricing and
deploying debt responsibly.

�ݺ�ߣ

2022-08-16-cloud-austin-tech-debt.pdf

More Related Content

2022-08-16-cloud-austin-tech-debt.pdf