Recently, our ZDLRA X8-2 was impacted by the EX80 issue, resulting in the loss of three disks within a very short time. Two of these disks were partner disks, causing the DELTA disk group to unmount and all customer data to be lost.
Oracle Support confirmed that recreating the DELTA disk group was not an option. We had to spend three days reimaging the entire ZDLRA system to restore operations.
What Is the EX80 Issue?
From Oracle documentation (Doc ID 2974254.1):
- Due to bug 35241309, Exadata High Capacity (HC) Storage Servers with certain hard disk drives may experience high failure rates.
- A signature symptom is multiple hard disk drive failures across different storage servers within minutes. If redundancy restoration has not completed, ASM disk group loss may occur, leading to database outages or requiring Data Guard failover or full database restores from backup.
Affected systems include:
- Exadata HC Storage Servers (X8/X8M) with 14TB drives W7214A520ORA014T and/or W7214A524ORA014T
- Exadata HC Storage Servers (X9M) with 18TB drive W7218A520ORA018T (preventive action recommended)
- Others with drives: 14TB W7214B520ORA014T, 10TB W7210B520ORA010T, 8TB W7208C520ORA8.0T
Our Experience
- Three disks failed between 04:14 and 05:17 morning across two storage cells.
- DELTA disk group, configured in normal redundancy, was lost due to simultaneous partner disk failures.
- Reimaged ZDLRA to version 25.1.7 and set DELTA disk group with high redundancy to mitigate risk.
Key Observations
1. ASM Redundancy Risk
From Oracle’s Zero Data Loss Recovery Appliance Guide:
- DELTA pool uses normal redundancy by default – only two copies of data are maintained.
This configuration is risky since losing two partner disks simultaneously means complete data loss.
2. Documentation Gaps
Oracle Doc ID 2974254.1 suggests EX80 affects versions for the hard disk drive firmware lower than A432/C432 for 14TB hard disk drives was supplied with the following Exadata versions:
- 25.1.0 through 25.1.1
- 24.1.0 through 24.1.8 (except 24.1.8.0.0.250130)
- 23.1.0 through 23.1.22
- 22.1.0 through 22.1.31
- 21.2 and all lower versions
However, our cells were running 24.1.10 and 25.1.7, and still hit EX80.
This means patching alone does not guarantee protection.
3. Exachk Limitations
Oracle recommends running ExaChk daily to detect EX80:
exachk -check 070E8C0B8BA9578CE0639812F50A82FC,070E8C0D41E5E8D2E0639912F50A5BF0
But:
- ExaChk did not detect all problematic disks in our case.
- Oracle support found two more failing disks after reviewing logs manually.
4. Oracle’s Recommendations
- Run latest Exachk daily for EX80.
- Apply latest patches (though not a full fix).
- Proactively replace all affected disk models (W7214A520ORA014T / W7214A524ORA014T).
- Use High Redundancy for all disk groups where possible.
Conclusion
- EX80 can cause severe data loss in Exadata X8/X9 and ZDLRA X8/X9 environments.
- Daily Exachk and latest patching help but do not eliminate risk.
- Best practice: Replace all affected drives and configure critical disk groups (e.g., DELTA) with High Redundancy.
Oracle DBA Insights 