🧚 👩🏽‍🔬 🚴🏿 Cara memulihkan Sentry setelah pembaruan yang gagal 🌄 🧝 🦑

Halo. Saya ingin berbicara tentang bagaimana pemulihan Sentry berjalan setelah pembaruan yang gagal.

Apa itu Sentry ?

Ini adalah open source, sistem pelacakan bug lengkap yang mendukung berbagai sisi server, browser, desktop dan bahasa dan lingkungan seluler asli, termasuk PHP, Node. js, Python, Ruby, C #, Java, Go, React, Angular, Vue, JavaScript dan banyak lagi.

Sedikit tentang betapa sederhana dan tidak terkendala semuanya rusak.

Kami tinggal di versi 9.0.0 dan sudah waktunya untuk memperbarui. Setelah merasakan antarmuka web 10.0.0, saya memutuskan untuk meningkatkannya. Ini adalah kesalahan terbesar!

Pembaruan berlangsung seperti biasa. Pertama, lanjutkan ke 9.1.2, lalu ke 10.0.0 (jika tidak, tidak mungkin). Kemudian saya menemukan bahwa pekerja tidak memulai. Alasannya adalah penambahan aplikasi dan dependensi baru.

Upaya untuk membangun aplikasi dan memasukkannya ke dalam k8 tidak berhasil.

Diputuskan untuk memutar kembali. Tapi disini saya kecewa. Setelah bermigrasi ke versi 10, sebagian data dipindahkan ke clickhouse dan dihapus dari Postgres.

Setelah 3 jam kerja dan keterlibatan ahli DBA, data dikembalikan ke kondisi kerja.

Bagaimana?

Kami membuat instance PG di sebelahnya, menghapus skema dari versi 9.0.0 di sana dan memulai pemulihan tabel. Tentu saja, ada beberapa kesalahan dan file.

Ceritanya tidak berakhir di situ. Saatnya meningkatkan ke versi 9.1.2.

Mengumpulkan gambar, jalankan:

sentry upgrade

dan kami melihat pesan yang luar biasa ini:

 ! I'm not trusting myself; either fix this yourself by fiddling
 ! with the south_migrationhistory table, or pass --delete-ghost-migrations
 ! to South to have it delete ALL of these records (this may not be good).
Exception in thread Thread-1 (most likely raised during interpreter shutdown)

Setelah pencarian yang lama, coba-coba, solusi ditemukan.

Anda perlu memulihkan skema dari versi saat ini.

Untuk melakukan ini, kami menggandakan repositori:

git clone https://github.com/getsentry/onpremise.git

Kami pergi ke cabang 9.1.2, karena cabang 9.0.0 sudah tidak ada lagi.

cd onpremise && git checkout tags/9.1.2

Selanjutnya, kami mengedit gambar di Dockerfile:

#cat Dockerfile 
ARG SENTRY_IMAGE
FROM ${SENTRY_IMAGE:-sentry:9.0.0}-onbuild

Kemudian edit file tulis

docker-compose.yml

# NOTE: This docker-compose.yml is meant to be just an example of how
# you could accomplish this on your own. It is not intended to work in
# all use-cases and must be adapted to fit your needs. This is merely
# a guideline.

# See docs.getsentry.com/on-premise/server/ for full
# instructions

version: '3.4'

x-defaults: &defaults
  restart: unless-stopped
  build:
    context: .
  depends_on:
    - redis
   # - postgres
    - memcached
    - smtp
  env_file: .env
  environment:
    SENTRY_MEMCACHED_HOST: memcached
    SENTRY_REDIS_HOST: redis
    SENTRY_POSTGRES_HOST: 'sentry.cl.ats'
    SENTRY_DB_USER: 'sentryDB'
    SENTRY_DB_NAME: 'sentry_user'
    SENTRY_DB_PASSWORD: 'sentry_passwd'
    SENTRY_POSTGRES_PORT: 5432
    SENTRY_EMAIL_HOST: smtp
  volumes:
    - sentry-data:/var/lib/sentry/files


services:
  smtp:
    restart: unless-stopped
    image: tianon/exim4

  memcached:
    restart: unless-stopped
    image: memcached:1.5-alpine

  redis:
    restart: unless-stopped
    image: redis:3.2-alpine

  #postgres:
  #  restart: unless-stopped
  #  image: postgres:9.5
  #  environment:
  #    POSTGRES_HOST_AUTH_METHOD: 'trust'
  #  volumes:
  #    - sentry-postgres:/var/lib/postgresql/data

  web:
    <<: *defaults
    ports:
      - '9000:9000'

  cron:
    <<: *defaults
    command: run cron

  worker:
    <<: *defaults
    command: run worker


volumes:
    sentry-data:
      external: true
    #sentry-postgres:
    #  external: true

Sedikit penjelasan. Karena saya memiliki instance database terpisah, saya melewatkan pembuatan melalui compose, dan volume sentry-postgres juga tidak diperlukan.

Setelah manipulasi ini, kami memulai perakitan:

./install.sh

Dalam prosesnya, kesalahan mungkin muncul, yang saya sebutkan di atas. Kami tidak memperhatikan dan setelah proses selesai, jalankan perintah berikut:

docker-compose run --rm web sentry django migrate --delete-ghost-migrations

Kesimpulan perkiraan

Creating network "onpremise_default" with the default driver
Creating onpremise_memcached_1 ... done
Creating onpremise_smtp_1      ... done
Creating onpremise_redis_1     ... done
21:30:26 [INFO] sentry.plugins.github: apps-not-configured
Running migrations for sentry:
- Nothing to migrate.
 - Loading initial data for sentry.
Installed 0 object(s) from 0 fixture(s)
Running migrations for nodestore:
- Nothing to migrate.
 - Loading initial data for nodestore.
Installed 0 object(s) from 0 fixture(s)
Running migrations for search:
- Nothing to migrate.
 - Loading initial data for search.
Installed 0 object(s) from 0 fixture(s)
Running migrations for social_auth:
- Nothing to migrate.
 - Loading initial data for social_auth.
Installed 0 object(s) from 0 fixture(s)
Running migrations for tagstore:
- Nothing to migrate.
 - Loading initial data for tagstore.
Installed 0 object(s) from 0 fixture(s)
Running migrations for jira_ac:
- Nothing to migrate.
 - Loading initial data for jira_ac.
Installed 0 object(s) from 0 fixture(s)
Running migrations for hipchat_ac:
- Nothing to migrate.
 - Loading initial data for hipchat_ac.
Installed 0 object(s) from 0 fixture(s)

Kemudian Anda dapat menjalankan peningkatan.

docker-compose run --rm web sentry upgrade

Stdout

Starting onpremise_smtp_1      ... done
Starting onpremise_memcached_1 ... done
Starting onpremise_redis_1     ... done
21:30:48 [INFO] sentry.plugins.github: apps-not-configured
Syncing...
Creating tables ...
Installing custom SQL ...
Installing indexes ...
Installed 0 object(s) from 0 fixture(s)

Synced:
 > django.contrib.admin
 > django.contrib.auth
 > django.contrib.contenttypes
 > django.contrib.messages
 > django.contrib.sessions
 > django.contrib.sites
 > django.contrib.staticfiles
 > crispy_forms
 > debug_toolbar
 > raven.contrib.django.raven_compat
 > rest_framework
 > sentry.plugins.sentry_interface_types
 > sentry.plugins.sentry_mail
 > sentry.plugins.sentry_urls
 > sentry.plugins.sentry_useragents
 > sentry.plugins.sentry_webhooks
 > sudo
 > south
 > sentry_plugins.slack

Not synced (use migrations):
 - sentry
 - sentry.nodestore
 - sentry.search
 - social_auth
 - sentry.tagstore
 - sentry_plugins.jira_ac
 - sentry_plugins.hipchat_ac
(use ./manage.py migrate to migrate these)
Running migrations for sentry:
- Nothing to migrate.
 - Loading initial data for sentry.
Installed 0 object(s) from 0 fixture(s)
Running migrations for nodestore:
- Nothing to migrate.
 - Loading initial data for nodestore.
Installed 0 object(s) from 0 fixture(s)
Running migrations for search:
- Nothing to migrate.
 - Loading initial data for search.
Installed 0 object(s) from 0 fixture(s)
Running migrations for social_auth:
- Nothing to migrate.
 - Loading initial data for social_auth.
Installed 0 object(s) from 0 fixture(s)
Running migrations for tagstore:
- Nothing to migrate.
 - Loading initial data for tagstore.
Installed 0 object(s) from 0 fixture(s)
Running migrations for jira_ac:
- Nothing to migrate.
 - Loading initial data for jira_ac.
Installed 0 object(s) from 0 fixture(s)
Running migrations for hipchat_ac:
- Nothing to migrate.
 - Loading initial data for hipchat_ac.
Installed 0 object(s) from 0 fixture(s)
Creating missing DSNs
Correcting Group.num_comments counter

Jadi, sirkuitnya dipulihkan. Anda dapat meningkatkan ke 9.1.2.

Untuk melakukan ini, di Dockerfile, ubah versinya menjadi 9.1.2:

ARG SENTRY_IMAGE
FROM ${SENTRY_IMAGE:-sentry:9.1.2}-onbuild

Dan jalankan install.sh

./install.sh

Jika semuanya dilakukan dengan benar, proses akan selesai tanpa kesalahan. Outputnya harus seperti ini:

Stdout

# ./install.sh 
Checking minimum requirements...

Creating volumes for persistent storage...
Created sentry-data.

.env already exists, skipped creation.

Building and tagging Docker images...

smtp uses an image, skipping
memcached uses an image, skipping
redis uses an image, skipping
Building web
Step 1/2 : ARG SENTRY_IMAGE
Step 2/2 : FROM ${SENTRY_IMAGE:-sentry:9.1.2}-onbuild
# Executing 4 build triggers
 ---> Running in 6c97f9fcaf63
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Removing intermediate container 6c97f9fcaf63
 ---> Running in 9e0f65ee3af6
Removing intermediate container 9e0f65ee3af6
 ---> Running in 09754c44319c
Removing intermediate container 09754c44319c
 ---> a12fa31c2675

Successfully built a12fa31c2675
Successfully tagged onpremise_web:latest
Building cron
Step 1/2 : ARG SENTRY_IMAGE
Step 2/2 : FROM ${SENTRY_IMAGE:-sentry:9.1.2}-onbuild
# Executing 4 build triggers
 ---> Using cache
 ---> Using cache
 ---> Using cache
 ---> Using cache
 ---> a12fa31c2675

Successfully built a12fa31c2675
Successfully tagged onpremise_cron:latest
Building worker
Step 1/2 : ARG SENTRY_IMAGE
Step 2/2 : FROM ${SENTRY_IMAGE:-sentry:9.1.2}-onbuild
# Executing 4 build triggers
 ---> Using cache
 ---> Using cache
 ---> Using cache
 ---> Using cache
 ---> a12fa31c2675

Successfully built a12fa31c2675
Successfully tagged onpremise_worker:latest

Docker images built.

Generating secret key...
Secret key written to .env

Setting up database...
Starting onpremise_smtp_1      ... done
Starting onpremise_redis_1     ... done
Starting onpremise_memcached_1 ... done
21:35:07 [WARNING] sentry.utils.geo: settings.GEOIP_PATH_MMDB not configured.
21:35:10 [INFO] sentry.plugins.github: apps-not-configured
Syncing...
Creating tables ...
Installing custom SQL ...
Installing indexes ...
Installed 0 object(s) from 0 fixture(s)
Migrating...
Running migrations for sentry:
 - Migrating forwards to 0472_auto__add_field_sentryapp_author.
 > sentry:0424_auto__add_field_integration_status
 > sentry:0425_auto__add_index_pullrequest_organization_id_merge_commit_sha
 > sentry:0425_remove_invalid_github_idps
 > sentry:0426_auto__add_sentryappinstallation__add_sentryapp__add_field_user_is_sent
 > sentry:0427_auto__add_eventattachment__add_unique_eventattachment_project_id_event
 > sentry:0428_auto__add_index_eventattachment_project_id_date_added
 > sentry:0429_auto__add_integrationexternalproject__add_unique_integrationexternalpr
 > sentry:0430_auto__add_field_organizationintegration_status
 > sentry:0431_auto__add_field_externalissue_metadata
 > sentry:0432_auto__add_field_relay_is_internal
 > sentry:0432_auto__add_index_userreport_date_added__add_index_eventattachment_date_
 > sentry:0433_auto__add_field_relay_is_internal__add_field_userip_country_code__add_
 > sentry:0434_auto__add_discoversavedqueryproject__add_unique_discoversavedqueryproj
 > sentry:0435_auto__add_field_discoversavedquery_created_by
 > sentry:0436_rename_projectdsymfile_to_projectdebugfile
 > sentry:0437_auto__add_field_sentryapp_status
 > sentry:0438_auto__add_index_sentryapp_status__chg_field_sentryapp_proxy_user__chg_
 > sentry:0439_auto__chg_field_sentryapp_owner
 > sentry:0440_auto__del_unique_projectdebugfile_project_debug_id__add_index_projectd
 > sentry:0441_auto__add_field_projectdebugfile_data
 > sentry:0442_auto__add_projectcficachefile__add_unique_projectcficachefile_project_
 > sentry:0443_auto__add_field_organizationmember_token_expires_at
 > sentry:0443_auto__del_dsymapp__del_unique_dsymapp_project_platform_app_id__del_ver
 > sentry:0444_auto__add_sentryappavatar__add_field_sentryapp_redirect_url__add_field
 > sentry:0445_auto__add_promptsactivity__add_unique_promptsactivity_user_feature_org
 > sentry:0446_auto__add_index_promptsactivity_project_id
 > sentry:0447_auto__del_field_promptsactivity_organization__add_field_promptsactivit
 > sentry:0448_auto__add_field_sentryapp_is_alertable
 > sentry:0449_auto__chg_field_release_owner
 > sentry:0450_auto__del_grouphashtombstone__del_unique_grouphashtombstone_project_ha
 > sentry:0451_auto__del_field_projectbookmark_project_id__add_field_projectbookmark_
 > sentry:0452_auto__add_field_sentryapp_events
 > sentry:0452_auto__del_field_releaseenvironment_organization_id__del_field_releasee
 > sentry:0453_auto__add_index_releasefile_release_name
 > sentry:0454_resolve_duplicate_0452
 > sentry:0455_auto__add_field_groupenvironment_first_seen
 > sentry:0456_auto__add_dashboard__add_unique_dashboard_organization_title__add_widg
 > sentry:0457_auto__add_field_savedsearch_is_global__chg_field_savedsearch_project__
 > sentry:0457_auto__add_monitorcheckin__add_monitor__add_index_monitor_type_next_che
 > sentry:0458_global_searches_data_migration
Saved Searchs: 100% |########################################################################################################################################################################| Time: 0:00:07
 > sentry:0459_global_searches_unique_constraint
 > sentry:0460_auto__add_field_servicehook_organization_id
 > sentry:0461_event_attachment_indexes
 > sentry:0462_auto__add_servicehookproject
 > sentry:0462_releaseenvironment_project_id
 > sentry:0463_backfill_service_hook_project
 > sentry:0464_auto__add_sentryappcomponent__add_field_sentryapp_schema
 > sentry:0464_groupenvironment_foreignkeys
 > sentry:0465_sync
 > sentry:0466_auto__add_platformexternalissue__add_unique_platformexternalissue_grou
 > sentry:0467_backfill_integration_status
 > sentry:0468_auto__add_field_projectdebugfile_code_id__add_index_projectdebugfile_p
 > sentry:0468_recent_search
 > sentry:0469_fix_state
 > sentry:0470_org_saved_search
 > sentry:0471_global_saved_search_types
 > sentry:0472_auto__add_field_sentryapp_author
The following content types are stale and need to be deleted:

    sentry | dsymobject
    sentry | dsymapp
    sentry | useridentity
    sentry | dsymsymbol
    sentry | dsymsdk
    sentry | globaldsymfile
    sentry | versiondsymfile
    sentry | projectdsymfile
    sentry | grouphashtombstone
    sentry | minidumpfile
    sentry | dsymbundle

Any objects related to these content types by a foreign key will also
be deleted. Are you sure you want to delete these content types?
If you're unsure, answer 'no'.

    Type 'yes' to continue, or 'no' to cancel: yes
Running migrations for sentry.nodestore:
- Nothing to migrate.
Running migrations for sentry.search:
- Nothing to migrate.
Running migrations for social_auth:
- Nothing to migrate.
Running migrations for sentry.tagstore:
- Nothing to migrate.
Running migrations for sentry_plugins.hipchat_ac:
- Nothing to migrate.
Running migrations for sentry_plugins.jira_ac:
- Nothing to migrate.

Synced:
 > django.contrib.admin
 > django.contrib.auth
 > django.contrib.contenttypes
 > django.contrib.messages
 > django.contrib.sessions
 > django.contrib.sites
 > django.contrib.staticfiles
 > crispy_forms
 > debug_toolbar
 > rest_framework
 > sentry.plugins.sentry_interface_types
 > sentry.plugins.sentry_mail
 > sentry.plugins.sentry_urls
 > sentry.plugins.sentry_useragents
 > sentry.plugins.sentry_webhooks
 > sudo
 > south

Migrated:
 - sentry
 - sentry.nodestore
 - sentry.search
 - social_auth
 - sentry.tagstore
 - sentry_plugins.hipchat_ac
 - sentry_plugins.jira_ac
Creating missing DSNs
Correcting Group.num_comments counter
Cleaning up...

----------------
You're all done! Run the following command get Sentry running:

  docker-compose up -d

Itu saja, versi 9.1.2 diinstal.

Kesimpulan.

Ini adalah kebenaran umum yang diketahui semua orang. Luangkan waktu Anda, buat backup dan periksa semuanya beberapa kali.

Terima kasih atas perhatian Anda!

Cara memulihkan Sentry setelah pembaruan yang gagal

More articles: