Title
Mobile Commons: RCA - 2021/01/14 - DB high load - Intermittent UI time outs
Information
Title: Mobile Commons DB high load – intermittent UI time outs
DATE, TIME AND DURATION
START TIME: 2022-01-14 3:20pm PM Eastern Time RESOLUTION TIME: 2022-01-14 4:20 PM Eastern Time
OVERVIEW
On January 14, 2022, Mobile Commons Support received reports of intermittent UI time-out errors, delays in mConnect calls, and general platform slowness.
ROOT CAUSE
On Jan 14, 2022, we started receiving an unexpected extremely high volume of API calls from our 3rd party vendor related to triggering the mConnect/outbound phone calls from the platform. These requests started to pile up and cause locking on the DB. During that time, the system was also trying to process high volume of MMS DLRs which added more load on the DB.
CORRECTIVE ACTION
The team started working immediately on cleaning any complex unnecessary queries on the DB. Additionally, we worked on restarting the passenger processes on the app servers once they have attempted to process a request for more than 5min to keep things running.
Keywords RCA import upload document RCA_2021.1.14 raexternalrequest
Solution ID
220518091940310
Last Modified Date
05/18/2022 09:29:12 AM
Renew Date
05/18/2023
Status
Published
Author
Taxonomy
- Mobile Commons > RCA-Root Cause Analysis > 2022
Collections
- UMM Support
Upland RightAnswers Portal - Version 2024R2