Menangkap bug pada klien: bagaimana kami menulis sistem kami untuk mengumpulkan kesalahan klien

Badoo . Badoo Bumble, - ( ), (Android iOS), UI.  Gelat. Error Tracking . 

, , , , .


:

  • , , , . 

  • , ;

  • , , .

β€” . , , , , .

β€” . Badoo : 

β€” -: - , ; 

β€” : ( , App Store Google Play). 

. - , , .

. , , QA-. 

Badoo : HockeyApp - JS-.

HockeyApp

HockeyApp , 2014 Microsoft , App Center. App Center : , , - Android- DexGuard mapping-, . ; , , - . 

Microsoft β€” 16 2019 , HockeyApp App Center. , DexGuard App Center 2019 , HockeyApp.

HockeyApp. Microsoft HockeyApp, , . .

JS-

JS-, , .

:

  • MySQL ( 10β€”20 );

  • ;

  • MySQL, Sphinx .

2017 -. β€” . - , - , - , .

, , . , 2014 , , . 

, .

  • .

  • . (, ) . , .

  • . , . .

  • β€” .

  • Β« ?Β», Β« ?Β».

  • - ( , , . .).

  • .

  • : , , , . .

  • Jira. Jira, .

  • Self-hosted. , . .

, , . 

SaaS-, : β€” . Bugsnag, TrackJS, Raygun, Rollbar Airbrake. , : , , HockeyApp. β€” .

Open-source-

open-source- . , .

Sentry , . ( 2018 ) :

  • ;

  • PostgreSQL, ;

  • , Python, β€” PHP Go;

  • (, Jira).

2018 Sentry, issue- , , β€” ClickHouse ( ). , . , , , . 

Gelato (General Error Logs And The Others), .

, , , , , , . : , , , .

.

, .

, .

, , , , . Jira .

:

, .

(1), (2) (3).

, .

, , β€” . , , , , - ().

, :

, , . , , :

  • A/B- β€” , -;

  • ;

  • ( );

  • email- .

, . :

  1. .

  2. .

  3. .

:

.

, , , β€” API .

API:

  • ;

  • , «» ( Β« Β», «» API);

  • .

?

, EPS (errors per second), , , .

- , . . , , , EPS ( iOS Facebook SDK 10 2020 ). , , .

, , β€” . : , - β€” , . 

, ( ), , .

?

  1. , , β€” , Redis RabbitMQ.

  2. Apache Kafka, , (, - ). Kafka () Sentry.

  3. LSD (Live Streaming Daemon). , . Badoo , .

: Β« ?Β» ( ) Β« ?Β» ( ).

: Elasticsearch ClickHouse.

Elasticsearch

:

  • ;

  • , ;

  • ;

  • UPDATE ( , , , );

  • DELETE ( , , );

  • API, .

, , Elasticsearch :

  • , ( SQL, (X-Pack) Open Distro Amazon);

  • JVM, Β« Β», β€” PHP Go (, , ; 6.8 7.5, (, ));

  • ; , , (, deflate- LZ4, CPU, ).

ClickHouse

:

  • , ;

  • MySQL- , , Elasticsearch;

  • , Elasticsearch.

2018 ClickHouse :

  • DELETE ( , ; ClickHouse , ( β€” ) );

  • UPDATE : ClickHouse , β€” ( GitHub β€” 2018 , );

  • ( RegExp, (full scan), ).

, ( ()). , , Elasticsearch.

, Elasticsearch ClickHouse, . . , Elasticsearch, , β€” .

, .

:

  • -;

  • .

( ) β€” . , warm- ( hot-warm-cold-).

JS-, - , , ( , , , ). , , attributes . 

Sentry . Sentry , tags , , extra β€” .

, , β€” . , , , , - . β€” .

.

- Android-

, Android- , :

  • (code shrinking β€” );

  • , (optimization β€” );

  • , , , - (obfuscation β€” ).

.

:

  • ProGuard ( ); 

  • DexGuard ProGuard ( );

  • R8 Google.

, - :

o.imc: Error loading resources: Security check required
  at o.mef.b(:77)
  at o.mef.e(:23)
  at o.mef$a.d(:61)
  at o.mef$a.invoke(:23)
  at o.jij$c.a(:42)
  at o.jij$c.apply(Unknown Source:0)
  at o.wgv$c.a_(:81)
  at o.whb$e.a_(:64)
  at o.wgs$b$a.a_(:111)
  at o.wgy$b.run(:81)
  at o.vxu$e.run(:109)
  at android.os.Handler.handleCallback(Handler.java:790)
  at android.os.Handler.dispatchMessage(Handler.java:99)
  at android.os.Looper.loop(Looper.java:164)
  at android.app.ActivityThread.main(ActivityThread.java:6626)
  at java.lang.reflect.Method.invoke(Native Method)
  at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:438)
  at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:811)

, . -, . , mapping.txt, . :

AllGoalsDialogFragment -> o.a:
    java.util.LinkedHashMap goals -> c
    kotlin.jvm.functions.Function1 onGoalSelected -> e
    java.lang.String selectedId -> d
    AllGoalsDialogFragment$Companion Companion -> a
    54:73:android.view.View onCreateView(android.view.LayoutInflater,android.view.ViewGroup,android.os.Bundle) -> onCreateView
    76:76:int getTheme() -> getTheme
    79:85:android.app.Dialog onCreateDialog(android.os.Bundle) -> onCreateDialog
    93:97:void onDestroyView() -> onDestroyView

, , mapping- β€” -.

(, , ), , , ProGuard ( DexGuard ) ReTrace, .

Android- Kotlin, : 

  • - ;

  • mapping- Ceph ( TeamCity);

  • -.

- iOS-

- iOS- , - ( - ). : - , .

Thread 0:
0   libsystem_kernel.dylib              0x00000001bf3468b8 0x1bf321000 + 153784
1   libobjc.A.dylib                     0x00000001bf289de0 0x1bf270000 + 105952
2   Badoo                               0x0000000105c9c6f4 0x1047ec000 + 21694196
3   Badoo                               0x000000010657660c 0x1047ec000 + 30975500
4   Badoo                               0x0000000106524e04 0x1047ec000 + 30641668
5   Badoo                               0x000000010652b0f8 0x1047ec000 + 30667000
6   Badoo                               0x0000000105dce27c 0x1047ec000 + 22946428
7   Badoo                               0x0000000105dce3b4 0x1047ec000 + 22946740
8   Badoo                               0x0000000104d41340 0x1047ec000 + 5591872

. -, - (dSYM), , , .

?

  • , , , macOS. , , Linux.

  • Symbolicator Sentry, ( , ). - , as is : , Rust . 

  • Symbolic Sentry, , Rust, C-ABI β€” FFI.

Golang , Symbolic cgo.

, , β€” . , .

, , , , - . (fingerprint), . -. , , ( ).

:

  • JS- message, type origin;

  • Android - - ( );

  • iOS - (, crashed -).

, .

, Sentry, . 

, , . 

, :




All Articles