Title

Mobile Commons: RCA - 2022/07/19 - DB Heavy load - 502 time-out errors

Information

TITLE: Mobile Commons | DB Heavy load – 502 time-out errors | 2022/07/19

DATE, TIME AND DURATION

START TIME: 2022/07/19 – 11:20 ET RESOLUTION TIME: 2022/07/23 – 14:34 ET DURATION: 0D 3H 14M

OVERVIEW

The Upland Mobile Commons team started seeing extreme platform slowness and intermittent 502 time-out errors on 7/19/2022 around 11:20 ET. Our cloud team started working immediately on investigating the root cause and restoring the system performance. The investigation showed that this was related to a heavy database load that affected one of the database nodes.

REMEDIATION

REMEDIATION PLAN:
- Short term: The cloud team started working immediately on replacing the affected DB node and restart all the backend processes to restore the platform performance.
- Long term: The cloud team is actively working on a long term infrastructure improvement process including switching to a new, more robust DB technology that would eliminate the performance issues we are currently seeing.
RCA ISSUE ID: [MC-10609]

Keywords RCA RCA_7.19.2022 raexternalrequest

Solution ID

220815160759250

Last Modified Date

08/15/2022 04:12:09 PM

Renew Date

08/15/2023

Status

Published

Author

bmagdy@uplandsoftware.com

Taxonomy

Collections

Upland RightAnswers Portal - Version 2024R2