Information
TITLE: Error when trying to access the platform
DATE, TIME AND DURATION
START TIME: 2025/04/17 – 10:00 ET RESOLUTION TIME: 2025/04/17 – 10:20 ET DURATION: 0D 0H 20M
OVERVIEW
On Thursday, April 17th, at approximately 10:00 AM ET, a product engineer was alerted to a platform issue that was causing login attempts to fail with an error. The engineering team was immediately engaged to investigate the issue and determine the root cause. Initial findings indicated that a change to database access had been made in production without undergoing the full verification process in staging. The issue was resolved shortly after by rolling back the change, restoring normal login functionality.
ROOT CAUSE & REMEDIATION
- ROOT CAUSE: The issue was traced to a human error. A change involving database access was made directly in the production environment without undergoing complete verification and testing procedures in the staging environment.
- REMEDIATION PLAN:
- Short Term: Once identified, the problematic change was rolled back, and normal login functionality was immediately restored.
- Long Term:
- Review and improve the internal process for production updates to make sure these are clearly documented, communicated and fully verified in the staging environment.
- Implementing monitoring and alerting for the application-DB connectivity.
- RCA ISSUE ID: [MC-15548]
Keywords RCA 2025 raexternalrequest
Solution ID
250505161927867
Last Modified Date
05/05/2025 04:25:31 PM
Renew Date
05/05/2026
Status
Published
Author
Taxonomy
- Mobile Commons > RCA-Root Cause Analysis
Collections
- UMM Self-Service
- UMM Support
Upland RightAnswers Portal - Version 2024R2