Read this article, paying attention to each application used in data abstraction. Be able to explain the purpose of each application and summarize holistic configuration.
Everything described thus far has custom configuration documents with very particular knowledge required to complete them. This was a challenge for my engineering team, as we sought to bring these technologies to a financial and health services enterprise with thousands of developers. We'd have to teach each developer multiple different formats, and if we changed formats, we'd have to retrain everyone. We were also faced with the challenge that we'd still have workloads outside our container orchestrator. So, we created our own abstraction around higher-order objects. We have created a system that allows a developer or administrator to describe an application from birth to death.
There are a couple types of documents that provide this capability. We use a namespace.yaml to allocate resources to a set of objects. This document must be approved by an individual with the right level of spending authority to cover the estimated costs of the resources requested. Once those resources are approved, any application or other object inside that namespace can utilize those resources until they're fully consumed.
apiVersion: v1
kind: Namespace
name: a-namespace
spec:
environments:
- name: dev
clientFacing: false
resources:
cpu: 2
memory: 4Gi
storage: 10Gi
- name: test
resources:
cpu: 6
memory: 12Gi
storage: 20Gi
A second level of document describes a specific object. There are multiple types, such as application, database, and document. Everything has a pipeline. These documents describe the resources required for the specific object, the relationship of the object to other named objects, the environments in which this application will run, the way the application should be built and tested, and how it should be deployed and run. This document is then converted into multiple documents related to specific technologies, like a Jenkinsfile for Jenkins and a Deployment for Kubernetes. These are then referenced to ensure that the expected state of each environment is maintained.
apiVersion: v1
kind: Application
name: application-name
metadata:
labels:
tier: frontend
spec:
build:
type: maven
runImageBase: tomcat7
environmentTemplate:
replicas: 1
resources:
min.memory: 256Mi
max.memory: 2Gi
environmentVariables:
- name: SHARED_ENV
value: 'shared value'
- name: ANOTHER_ENV
value: 'another value'
ports:
- name: https
port: 443
volumes:
- name: shared-data
emptyDir: {}
mountPath: /var/lib/pipeline_data
connections:
- name: authn
environments:
- name: dev
environmentVariables:
- name: ENVIRONMENT_SPECIFIC
value: 'dev value'
This abstraction allows our developers to stay focused on creating business value, while a central team can utilize a single document interface to move an application from GitLab to production. Our tools can now be changed as needed without any changes from developers. It is the central team's responsibility to maintain the contract through these documents so that the developer experience doesn't change as tools change. Because this system has worked so well for us, we hope to open source it in the near future.