Ajavon scored nine points in the decisive third quarter and McCoughtry added eight. Atlanta trailed 36-31 at halftime, but outscored Mystics 21-9 in the third as Washington went without a field goal for 5-plus minutes.
Sancho Lyttle had 12 points, 10 rebounds and five steals for Atlanta (4-6). McCoughtry, the WNBA's second-leading scorer, had 21 points in Atlanta's 64-61 win over Washington on June 12.
Ivory Latta scored 18 points for Washington (5-3), which lost for the first time on the road this season. Emma Meesseman added 14 points and seven rebounds.
Atlanta played without Tiffany Hayes, the team's second-leading scorer with 12.2 points per game, and Aneika Henry, who are on international duty.