loader image
Skip to main content
If you continue browsing this website, you agree to our policies:
x
Completion requirements

Read this article, paying attention to each application used in data abstraction. Be able to explain the purpose of each application and summarize holistic configuration. 

Creating a holistic configuration

Everything described thus far has custom configuration documents with very particular knowledge required to complete them. This was a challenge for my engineering team, as we sought to bring these technologies to a financial and health services enterprise with thousands of developers. We'd have to teach each developer multiple different formats, and if we changed formats, we'd have to retrain everyone. We were also faced with the challenge that we'd still have workloads outside our container orchestrator. So, we created our own abstraction around higher-order objects. We have created a system that allows a developer or administrator to describe an application from birth to death.

There are a couple types of documents that provide this capability. We use a namespace.yaml to allocate resources to a set of objects. This document must be approved by an individual with the right level of spending authority to cover the estimated costs of the resources requested. Once those resources are approved, any application or other object inside that namespace can utilize those resources until they're fully consumed.

apiVersion: v1
kind: Namespace
name: a-namespace
spec:
  environments:
  - name: dev
    clientFacing: false
    resources:
      cpu: 2
      memory: 4Gi
      storage: 10Gi
  - name: test
    resources:
      cpu: 6
      memory: 12Gi
      storage: 20Gi

A second level of document describes a specific object. There are multiple types, such as application, database, and document. Everything has a pipeline. These documents describe the resources required for the specific object, the relationship of the object to other named objects, the environments in which this application will run, the way the application should be built and tested, and how it should be deployed and run. This document is then converted into multiple documents related to specific technologies, like a Jenkinsfile for Jenkins and a Deployment for Kubernetes. These are then referenced to ensure that the expected state of each environment is maintained.

apiVersion: v1
kind: Application
name: application-name
metadata:
  labels:
    tier: frontend
spec:
  build:
    type: maven
    runImageBase: tomcat7
  environmentTemplate:
    replicas: 1
    resources:
      min.memory: 256Mi
      max.memory: 2Gi
    environmentVariables:
      - name: SHARED_ENV
        value: 'shared value'
      - name: ANOTHER_ENV
        value: 'another value'
    ports:
      - name: https
        port: 443
    volumes:
      - name: shared-data
        emptyDir: {}
        mountPath: /var/lib/pipeline_data
    connections:
      - name: authn
  environments:
    - name: dev
      environmentVariables:
        - name: ENVIRONMENT_SPECIFIC
          value: 'dev value'

This abstraction allows our developers to stay focused on creating business value, while a central team can utilize a single document interface to move an application from GitLab to production. Our tools can now be changed as needed without any changes from developers. It is the central team's responsibility to maintain the contract through these documents so that the developer experience doesn't change as tools change. Because this system has worked so well for us, we hope to open source it in the near future.