Halo, Habr! Nama saya Sasha Kozlov, saya telah mengembangkan infrastruktur dan administrasi sistem di Avito selama tiga setengah tahun terakhir. Saya akan memberi tahu Anda bagaimana kami meningkatkan dan memodernisasi pekerjaan kami dengan kode infrastruktur dari waktu ke waktu dan membawanya ke tingkat yang baru secara kualitatif.
Kami bertanggung jawab atas siklus penuh manajemen peralatan: mulai dari pengadaan dan pemasangan di DC hingga pengiriman ke pengguna akhir. Ini tentang pelajaran yang telah kami pelajari selama beberapa tahun terakhir, bekerja dengan ribuan peralatan dan beberapa ribu konfigurasi.
Kami akan berbicara tentang alat IaC "generasi pertama" seperti Ansible, Chef, Salt, Puppet. Jika Anda berurusan dengan infrastruktur lokal, dan Anda tidak memiliki cloud sendiri pada virtualisasi, kemungkinan besar Anda sudah menggunakan salah satu alat ini.
Puppet. - , , Puppetlabs . , , , , . , β .
, . , . - push-, - pull, ansible pull-. β Python, Jinja- YAML-, β DSL Ruby. : , PuppetDB, , .
, . git revert, , . , , . β β , 0 β β. .
, Puppet. - β , , . , :
- , ;
- ;
- CI-, ;
- ;
- , , ;
- «» Configuration Drift.
, , . β . .
. : , , .
Puppet , . , .. pull-. β . 30 , , , Puppet . , event-driven , Salt, .
, . stateful-, , . .
: Puppet stateful-, , . , DNS- Puppet, . , , , API.
, . , . , , git . . .
: control repo . git-. , Puppet-, "The roles and profiles method".
Control repo β . , CI- - . β , . β , Β« ?Β». : k8s- , Kafka , ClickHouse- ..
β , . - . Puppet -, , . , , , .
control repo, .
β , , , . control repo, semver . 50 : , , Kubernetes .
, , Puppet. . , , . , control repo?
, , . control repo , . , . , , . Docker, . control repo, . , .
, , :
- Building a Functional Puppet Workflow Part 2: Roles and Profiles
- Roles and Profiles in a Control Repo?
- Workflows Evolved: Even Besterer Practices
- Profiles and the Path to Hiera Data
- Puppetlabs Best Practices Docs
- control repo, . , , , :
External Node Classifier
, , β , ?
node definitions , , . , - :
node /^avi-ceph(2[1-9]|3[0-9]|4[0-9]|5[0-9]|6[0-9]|7[0-9]|8[0-9]|9[0-9])/ {
...
}. β , . , , . Puppetlabs External Node Classifier.
External Node Classifier β , , . , . ENC , top-scope variables. control repo , , , node definition:
node default {
include base # , control repo
if $::role != '' {
notify{ "Node ${::fqdn} has role ${::role}": loglevel => info }
include "role::${role}"
} else {
notify{ "Node ${::fqdn} has no role": loglevel => warning }
}
}ENC , . CMDB, . CMDB netbox Digital Ocean, . razor, , , -.
CMDB - , 15-20 , , .
«» ? , Configuration Drift, . Configuration Drift β , , . β , , , - Puppet.
, , , . , . , k8s-.
, API-. netbox API, . PXE netboot- Debian, preseed' , API, IPMI Redfish API. , , , .
, , . , .
. : , , , , .
, β , . β . . βwrite-onlyβ , .
3.7 6 Puppet, , . :
- .
- -.
:
- , ..
- , , duplicate resource declaration. : , , .
- Docker-, , inspec.
Kubernetes Puppet "Kubernetes The Hard Way", .
, , smoke-, . , , .
, Beaker , , -, . ~~ smoke-: ~~ , .
, , :
context 'application deployment' do
it 'can deploy an application into a namespace and expose it' do
shell('systemctl restart kubelet')
shell('count=0;
while [[ $(kubectl get pods -n tiller -l name=tiller -o \'jsonpath={..status.conditions[?(@.type=="Ready")].status}\') != "True" ]];
do
if [[ count -gt 180 ]]; then
break;
fi;
sleep 1 && ((count++));
done')
shell('kubectl create -f /tmp/nginx.yaml', acceptable_exit_codes: [0]) do |r|
expect(r.stdout).to match(%r{namespace/nginx created\nconfigmap/my-nginx-config created\ndeployment.apps/my-nginx created\nservice/my-nginx created\n})
end
end
it 'can access the deployed service' do
shell('count=0;
while [[ $(kubectl get pods -n nginx -l run=my-nginx -o \'jsonpath={..status.conditions[?(@.type=="Ready")].status}\') != "True" ]];
do
if [[ count -gt 180 ]]; then
break;
fi;
sleep 1 && ((count++));
done')
shell('curl --connect-timeout 1 --retry-delay 1 --retry-max-time 300 --retry 150 -s --retry-connrefused 10.100.10.5', acceptable_exit_codes: [0]) do |r|
expect(r.stdout).to match %r{Welcome to nginx!}
end
end
end, . CI .
CI PR. PR : . , , , CI PR, , .
, :
- puppet-syntax
- puppet-lint
- rspec-puppet
- puppetlabs-spec-helper
- test-kitchen, Puppet: kitchen-docker-puppet-example
- beaker
-:
Development kit
, CI/CD β , workflow.
- β PDK, . , , . PDK control repo Kitchen Docker. PDK Beaker, .
, :
- , : , CI .
- , , -.
- , -, .
- control repo, , .
- .
- .
development tool, CI , .
(puppet-rspec, puppet-linter, test-kitchen) , . , , .
, , Β« Β». , .
, , . , β . : Forge. Puppet Forge , .
Puppet . , : metadata.json Puppetfile. , - . . librarian-puppet, , r10k, . , . β , : acceptance- -.
. , git-, :
mod 'dba-clickhouse',
:git => 'ssh://git@github.com/iac/dba-clickhouse.git',
:ref => '1.2.2'
mod 'dba-kafka',
:git => 'ssh://git@github.com/iac/dba-kafka.git',
:ref => '1.2.0', , . Puppet Forge, , librarian-puppet.
semver, , . , , β :
# Puppetfile
mod 'arch-puppetserver', '0.20.5' #
mod 'arch-vault', '~> 2.1' #
mod 'si-lxc' # , . , , , . :
[22:39:43] in dba-control on ξ production via ruby-2.5.1 at οΈ unstable
$ iack dep show
[] Collecting modules metadata
FULL NAME | CURRENT VERSION | LATEST VERSION | OUT OF DATE?
---------------------|-----------------|----------------|---------------
si-lxc | latest | 0.3.2 | N/A
si-base | latest | 1.3.1 | N/A
petems-hiera_vault | v0.4.1 | | Major
arch-vault | 2.1.0 | 2.1.0 | No
dba-postgresql | 0.1.2 | 0.1.3 | Tiny
dba-pgbouncer | 0.4.0 | 0.5.1 | Minor
si-grub | 0.1.0 | 0.1.0 | No
si-collectd | 0.2.3 | 0.2.4 | Tiny
si-confluent | 0.3.0 | 0.3.0 | No
dba-redis | 0.2.3 | 0.2.3 | No
dba-collectd_plugins | latest | 0.2.0 | N/A
dba-mongodb | 0.2.1 | 0.2.1 | No
dba-patroni | 0.1.4 | 0.2.4 | Minor
dba-cruise_control | 0.1.1 | 0.1.2 | Tiny
dba-lxd | 0.7.0 | 0.7.0 | No
dba-clickhouse | 1.2.1 | 1.2.2 | Tiny
dba-zookeeper | 2.0.0 | 2.0.0 | No
si-td_agent | 0.1.0 | 0.1.0 | No
dba-kafka | 1.1.6 | 1.2.1 | Minor
arch-puppetserver | 0.20.1 | 0.20.2 | Tiny
pcfens-filebeat | 4.1.0 | 4.4.1 | Minor
KyleAnderson-consul | 5.0.3 | 6.0.1 | Major
puppetlabs-apt | 6.3.0 | 7.4.2 | Major
puppetlabs-stdlib | 5.2.0 | 6.3.0 | Major:
- Managing environment content with a Puppetfile
- puppet-forge-server β ruby Sinatra, Puppet Forge
- librarian-puppet
code style
code style ? β , . , , , . review β , .
Puppet . The puppet language style guide . Puppet-lint, CI, .
, . control repo:
development kit, «»:
β
, . Puppet Hiera Vault. hiera-backend, vault hiera_lookup.
-, , - . , :
$token_data = vault::secret_field('tokens.csv', 'data')token_data 'data' tokens.csv, Vault. Vault , , Hiera:
$ vault-util ls puppet/arch/
common/
nodes/
roles/ , , common. β roles/ nodes/.
β , -, , , . TLS- .
, , , . , Puppet:
Canary
Puppet , control repo. control repo , . ENC .
, . , . β - .
, , , .
Puppet
Puppet, , , . DSL β Ruby. Hiera β , . , . Hiera , , .
Puppet β , , . . , : , Hiera, . - , , .
Puppet Ruby , . , pet project. , , β . : , , , .
β . , pull-. , , . , , pull push. Bolt, Puppetlabs , , PuppetDB, .
, , :
- , .
- .
- .
- .
- CI .
- , workflow .
- External Node Classifier .
- .
- .