Laporan Andrei Borodin menceritakan bagaimana mereka memperhitungkan pengalaman penskalaan PgBouncer dalam desain penarik senyawa Odyssey saat diluncurkan ke produksi. Selain itu, kami akan membahas fungsi puller apa yang ingin kami lihat di versi baru: penting bagi kami tidak hanya untuk memenuhi kebutuhan kami, tetapi juga untuk mengembangkan komunitas pengguna Odyssey .
Video:
Halo! Nama saya Andrey.
Di Yandex, saya mengembangkan database open source. Dan hari ini kami memiliki topik tentang koneksi pooler koneksi.
Jika Anda tahu cara memanggil koneksi pooler dalam bahasa Rusia, beri tahu saya. Saya benar-benar ingin menemukan istilah teknis yang baik yang harus mapan dalam literatur teknis.
, connection pooler . - , , , Postgres . ( HighLoad++ 2019) Postgres. , , , , , , . , .
Postgres. . . , Postgres.
– primary , ( ), , .
– Postgres, Postgres connection pooler. Connection pooler fencing .
connection pooler?
Postgres . , – , Postgres. , .
, Postgres , procArray. . procArray , . , . , .
3 :
- .
- .
- , . . .
, pooler . PostgreSQL Professional . , . . application-side pool proxy pool.
Application-side pool – . : .
, , .
, , -. client side pooling . – 10 000 . , .
proxy poolers, poolers, . poolers. poolers + . Pgpool Crunchy-Proxy.
, , . , poolers pooling, . . , .
, PgBouncer, transaction pooling, . . .
– . .
, , . loopback - .
, application_name_add_host. Bouncer IP- application_name. application_name .
, – , – , . – application_name, , .
, Bouncer pool, . . , .
? , C++ - , , . 20 000 , . .
, , Bouncer, , . . pool.
Postgres, . . .
, . PgBouncer , . : , , , , , - . . , , .
, .
, Bouncer . . , , pooler.
, PgBouncer’.
https://lwn.net/Articles/542629/
Bouncer.
, TCP- Bouncers. TCP- round-robin’ .
, . . , Bouncer, idle- Bouncer’.
, 3 Bouncers 100 %. Bouncers. ?
TLS. . Postgres TLS TLS, , , TLS handshake .
, . primary , – . . . - primary , -. TLS.
TLS handshake Bouncer, . - . retry exponential backoff, .
16 PgBouncer, 16 100 %.
PgBouncer. , Bouncer. Bouncers TCP handshake, Bouncers pooling, , .
. 18 Bouncers . . , DevOps , , .
, open source, Bouncer . , , PgBouncers . pull request .
https://www.postgresql.org/docs/current/libpq-cancel.html
https://github.com/pgbouncer/pgbouncer/pull/79
. Postgres , - . TCP-reset, . . . Bouncer? . . , , Bouncer , , .
upstream Bouncer’.
, connection pooler, , , , , .
. TLS-.
, Machinarium, . libpq, , : « . IO, , IO ». . . " N, N ", " SYNC, ". , . , . , , (green thread-). , , .
, TCP accept round-robin' workers TPC-.
. cache-friendly.
, , TCP-stack.
, , Odyssey CANCEL ROLLBACK , . . , Odyssey , , .
. application_name_add_host. , , .
.. managed PostgreSQL connection pooler, , . . , , . Bouncer .
.
, . , , , . , connection pooler Odyssey, .
Odyssey PgBouncer. , . - , pull request, issue GitHub, . PgBouncer .
, , error forwarding. , . , , , .
, 100%- PgBouncer. , Bouncer, .
Odyssey.
https://github.com/yandex/odyssey/pull/66
, «Pause /Resume». . Postgres, connection pooler, pg_upgrade, resume. , . . , . ( )
https://github.com/yandex/odyssey/pull/73 —
, PgBouncer, SCRAM Authentication, , .. – .
Odyssey, .
Odyssey, . Kiwi – Postgres’ . . . proto 3 Postgres – , , . Kiwi.
Machinarium – . Machinarium . , , 15 .
Odyssey. , coroutines. accept TCP- workers.
worker . crone- , pool.
Odyssey Postgres. install-check Bouncer Odyssey, div. , Bouncer Odyssey.
, , . Odyssey.
, - : Postgres + Odyssey, PgBouncer + Odyssey, Odyssey + Odyssey , , , Odyssey - , - , .
Odyssey production. , , . , . . , . , production , PostgreSQL Professional , memory leak. , . .
, connection pooler TLS- TLS-. .
Bouncer Odyssey pcache, pcache , Odyssey . , . , 20 000 .
Pluggable Authentication Method – lunux’ . PgBouncer , PAM PgBouncer, PAM.
. . ?
, , PAM- PAM-, PAM- PAM-. , . , .
, , accept . worker pool, TLS handshake.
, 20 000 , . libpq . 3 .
, , retry.
, PgBouncer , throttling TCP-, accept.
, , , , . , handshake , . - , .
Roadmap
Odyssey? ?
2019 .
roadmap Odyssey :
- SCRAM PAM authentication.
- forward standby.
- online-restart.
- .
roadmap , . . , .
forward read-only queries to standby? , . failover switchover. - - . , -, .
, Postgres, 10- session_attrs. , : . , , session_attrs.
, . - , . , , Odyssey , .
Odyssey primary. , , , , . .
, open source. , , 2,5 PgBouncer. Odyssey.
prepared statement. prepared statement . -, SQL-, «prepared». SQL- SQL Bouncer. overkill, , . . . SQL-.
prepared statement proto3. , , prepared statement, . , - prepared statements. , - .
, - , , prepared statements , , . . prepared statement.
, , prepared statement, . , , , , , , , .
, . , PgBouncer. . – : - , - – . .
, , , , .
– 1.0 ( 1.1). , Odyssey 1.0rc, . . release candidate. , , , memory leak.
1.0? Odyssey . , 1 000 000 , , , 1.0.
, 1.0 SCRAM. , production , SCRAM, . , , .
pull request. , Bouncer. . , - , .
, . !
application_name, , transaction pooling Odyssey?
Odyssey Bouncer?
Odyssey. Bouncer .
.
, ?
, . , application_name. , . . , .
, , ! ! , Odyssey . . multi data-source , Odyssey , . . slave, failover .
, , , . storages. . , , failover, pg_recovery. , . - ? . . , . . , C, .
, adoption . , . . , .
. , . , , , . , , , . , , , , , . . . , , .
, . pcache , , real cache , , , .
- , , , , . . .
. , . , , , 10 . , .
, .
. . , , . , .
Nginx memiliki opsi seperti itu slowly startdi cluster untuk server. Dan dia secara bertahap meningkatkan beban.
Ya, ide bagus, kami akan mencobanya saat kami sudah melakukannya.