minSdk gap, metrics coverage) compound, and an independent A/V lab confirmed slow-join is structural to Zoom, not a build artefact.Leadership asked us to evaluate whether a third-party SDK could be plugged in as the media layer for P2P calls while in-house work continues in parallel. Two candidates were on the table — Microsoft (server-side, separate track) and Zoom Video SDK (client-side, this evaluation). Plan: integrate Zoom alongside WhatRTC, run an A/B in the AE-PH corridor, let the data decide.
Over the month we built a POC (zoom_sdk_integration_poc branch in botim-android, ~10 Kotlin/Java files, local JWT auth server), ran a live two-device test in UAE, decompiled the AAR to verify Zoom's diagnostic claims, and audited the metrics surface against Botim's bad-call success matrix. In parallel, TestDevLab — an independent A/V lab whose clients include Zoom itself, Microsoft, Discord, Twilio, RingCentral — published a competitive A/V analysis comparing Botim, Comera, WhatsApp and the Zoom consumer app. Both data sources converged on the same conclusion.
| Metric | Botim threshold | Zoom Lite 2.4.6 observed | Verdict |
|---|---|---|---|
| Time to Ringing (caller tap → callee rings) | < 1000 ms | ~6 s | Fail |
| Time to Audio (callee Accept → first audio) | < 500 ms | ~5 s | Fail |
| Time to Video (callee Accept → first video) | < 1000 ms | ~5 s | Fail |
joinSession → onSessionJoin (raw SDK time) | n/a | 4.4–6.0 s | Inside Zoom's cloud handshake |
4.4 s with zoom.us resolving in 17 ms; callback dispatch confirmed on Main thread via decompiled ZoomVideoSDKDelegateHelper; approach (b) confirmed in ZoomSessionBridge.kt. None of the three explains the gap.
| Test condition | Metric | Botim | Comera | Zoom | Botim vs Zoom | |
|---|---|---|---|---|---|---|
| Changing Combined Video p.69 | Time to Ringing | 1.06 s | 0.82 s | 5.60 s | 1.65 s | 5.3× faster |
| Time to Video | 0.82 s | 1.31 s | 5.60 s | 1.44 s | 6.8× faster | |
| Time to Audio | 0.50 s | 1.37 s | 2.85 s | 0.93 s | 5.7× faster | |
| Changing Packet Loss p.77 | Time to Ringing | 1.03 s | 0.88 s | 5.80 s | 1.81 s | 5.6× faster |
| Time to Video | 0.91 s | 1.35 s | 5.80 s | 1.47 s | 6.4× faster | |
| Time to Audio | 0.48 s | 1.29 s | 4.81 s | 0.93 s | 10.0× faster | |
| Changing Combined Audio (iOS → Android) p.114 | Time to Ringing | 1.01 s | 0.85 s | 1.21 s | 5.50 s | on par |
| Time to Audio | 1.32 s | 1.14 s | 1.99 s | 3.80 s | 1.5× faster |
minSdk lockout| Item | Value |
|---|---|
Botim minSdk | 21 (Android 5) |
| Zoom Lite 2.4.6 / Full 2.5.5 required | 28 (Android 9) |
| Botim Android < 9 callers (24 h) | ~29 K DAU |
| ↳ Android 8.x share of that cohort | 86 % |
| Implication | Zoom is additive, never a replacement — WhatRTC stays bundled and maintained indefinitely. |
| Metric family | Botim | Zoom Lite |
|---|---|---|
| Total quality metrics tracked | 185 | ~75 |
onQOSStatisticsReceived push stream | — | Absent in AAR |
| Audio bitrate (send/recv) | ✓ | Missing |
| Encode/decode time per frame | ✓ | Missing |
| Jitter buffer / packet send delay | ✓ | Missing |
| Per-user RTT / jitter / loss / bw / codec | ✓ | Missing |
| Per-user audio metrics | ✓ | Missing |
onQOSStatisticsReceived requires switching to the Full SDK, which re-introduces the binary-size cost that Lite was specifically chosen to avoid. Lite's standalone ZoomVideoSDKVideoStatisticInfo exposes only width/height/fps/bpf/networkStatus.
2.4.6 (Android, ~62 MB), Lite 2.5.1 (iOS, ~105 MB), plus Lite Full 2.5.5 analysed for metric coverage.| Date | What happened |
|---|---|
| Apr 6 | Task kicked off. Opening meeting with Zoom team — open questions on architecture, callbacks, metrics, JWT, region routing. |
| Apr 8 | Analysed Zoom Video SDK v2.5.5 (Android): ~75 quality metrics vs Botim's 185, minSdk 28 vs Botim's 21, 70+ delegate callbacks. Local POC project started. |
| Apr 9 | Metrics inventory complete — material metrics gap confirmed. |
| Apr 10–15 | Quality comparison plan written: A/B success matrix (audio/video/connection bad-call thresholds, MOS, ratings). Document shared with Zoom team. |
| Apr 20–21 | Integration plan and design started. Risks logged: large SDK binary size; key bad-call tracking metrics missing in Zoom. |
| Apr 23 | Zoom Video SDK Lite 2.4.6 Android package received and integrated into zoom_sdk_integration_poc branch. JWT auth server wired up locally. |
| Apr 27 | First end-to-end live test in UAE/Dubai. Observed: 6.0 s joinSession → onSessionJoin on caller, 10.7 s onUserJoin. Encrypted SDK logs shared with Zoom. |
| May 5 | Zoom engineering replied with three claims (DNS, Main thread, approach (b)). Verified each on our side — all ruled out. DNS probe re-ran 4.4 s with zoom.us at 17 ms. |
| May 5 | Zoom announced "two-step join" SDK arriving May 8 and proposed workshop May 6. Botim position: no further engineering cycles until Zoom shares benchmark numbers proving thresholds clear in AE region. |
| May 6 | Joint session with Zoom engineering — they confirmed inherent join-time slowness; we requested the new SDK with benchmark data. |
| May 7 | VoIP team decided to pause implementation, based on the connect-time blocker, the minSdk lockout, the metrics-surface gap, and TestDevLab's independent confirmation. |
zoom_sdk_integration_poc on botim-android — preserved, no deletions planned. ~10 Kotlin/Java files under bot_business/voip_module/voip/src/main/java/im/thebot/voip/zoom/.Zoom_SDK_Integration_Final_Report.md · TestdevLab Report.pdf (118 pp) · Zoom_vs_Botim_Quality_Comparison_Plan.md · zoom_sdk_debug_logs.zip (CSV30-encrypted, both devices) · zoomsdk_delay.mp4 · Zoom sdk bundle/ (Lite Android 2.4.6 + iOS 2.5.1 binaries).
joinSession → onSessionJoin inside Botim's thresholds, on a packaged SDK we can integrate. Until then this evaluation does not advance.