Incident Report - 2019/09/18
On September 18, 2019, Duo’s Engineering Team was made aware of an issue with on-premises Active Directory syncs that resulted in some groups losing all of their members during sync. In addition, attempts to remove and re-add these groups from the directory sync page would fail. Depending on customer configuration, some removed users would have been unable to authenticate using Duo.
The root cause of this error was determined to be a bug introduced in Duo’s mid-September release, which began deploying to customers at 3 p.m. EDT on September 17, 2019. This bug was introduced when fixing an issue that affected syncing users from Active Directory that had leading or trailing whitespace in their usernames.
A fix for the issue has been added to the above-mentioned mid-September release and was redeployed to all affected customers, and was finished by 11:07 a.m. EDT on September 18th. Duo engineers also ran corrected directory syncs for all potentially affected customers in advance of their scheduled daily syncs. This effort was completed by 3:37pm ET. Duo’s Engineering team has also enhanced automated test coverage in this area to prevent this issue in the future.