Lessons from running a YBYR team

DevOps

DevOps (a clipped compound of "development" and "operations") is a software engineering culture and practice that aims at unifying software development (Dev) and software operation (Ops)

You build it, You run it

Giving developers operational responsibilities has greatly enhanced the quality of the services , both from a customer and a technology point of view. The traditional model is that you take your software to the wall that separates development and operations, and throw it over and then forget about it. Not at Amazon. You build it, you run it. This brings developers into contact with the day-to-day operation of their software. It also brings them into day-to-day contact with the customer. This customer feedback loop is essential for improving the quality of the service.

- A conversation with Werner Vogels - 30/Jun/2006

https://queue.acm.org/detail.cfm?id=1142065

You build it, You run it

You build it, you run it is the principle that development teams operate their own product.

Capacity Plan

New skill set

It takes time

It's an investiment

People want to run services

Autonomy and Accountability

It is easier said than done

Autonomy requires aligment

Autonomy is not free

Running Services

It is easy, I will automate it...

There is a lot we don't know

Dev analogy

Ops analogy

Out of comfort zone

Team Response so far

Operational Model

Google SRE Book

Services/Components

OR

Service Level Definitions

  • SLI = Service Level Indicators
  • SLO = Service Level Objectives
  • SLA = Service Level Agreement

Monitoring

  • Whitebox

    Monitoring based on metrics exposed by the internals of the system, including logs, interfaces like the Java Virtual Machine Profiling Interface, or an HTTP handler that emits internal statistics.
  • Blackbox

    Testing externally visible behavior as a user would see it.

On-call

Developers on-call

Incident Management

Testing

graph LR tester ==> deploy subgraph Engineering dev["Development"] ==> tester["Tester"] tester ==> VTE end subgraph Ops deploy["Deploy"] end

In the pipeline we trust

graph LR subgraph Engineering dev["Development"] ==> pl["Pipeline"] pl ==> deploy["Deploy"] end

Feature Toggles

Release Strategy

Release Monitoring

Done Definition

Working software in production

Cross Functional Team

Cross Functional Teams

Tuckmans' model

Storming is harder

T-Shaped People

The one required Skill

Coding

Everything as code

Code is everything

Pipeline

Multiple Pipelines

I mean MANY Pipelines

Pipelines Overview

  • Application Code
  • Database
  • Middleware
  • Infrastructure
  • Configuration

Break Glass

Thank you


www.roger-almeida.com/talks/ybyr/