Jonathan Schoop and Adam Jones connected in a four-run eighth inning, and Baltimore completed a two-game sweep of the rain-shortened series with a 5-3 win Wednesday night.
The Orioles lead the majors with 152 home runs, are 35-10 when hitting more than one and 58-25 when going deep at least once.
After Schoop tied it with a drive off Dellin Betances, Shawn Kelley (2-4) gave up a single and a walk before Jones hit a shot into the bullpen area beyond the center-field wall.
"When we're down by one we always feel we're just one pitch away," Jones said. "So that happened, and once we tie the game all the marbles are up. First one to score one wins."
Jones made it happen. As the ball dropped over the fence for his 23rd home run, he raised his right arm in triumph as he rounded first base.
"Big spot, big home run," Jones said. "Nothing wrong with showing a little emotion here and there."
Darren O'Day (4-1) worked the eighth and Zach Britton gave up a run in the ninth en route to his 25th save.
It was the eighth straight series win for the AL East-leading Orioles, who increased their margin over the fading third-place Yankees to eight games. Baltimore is 8-3 against New York this season, outscoring the Yankees 60-31.
"It's tough right now because they are hitting very good," New York's Francisco Cervelli said. "They've got a couple of hitters who are really hot right now."
Yankees manager Joe Girardi missed the finish after being ejected in the seventh inning by home plate umpire Gerry Davis. Girardi was furious after Davis called New York's Stephen Drew out for running in the baseline on his way to first base.
"Yeah, well, Gerry was wrong," Girardi said.
Girardi's second ejection of the year came with New York ahead 2-1 on the strength of Cervelli's second home run of the year, a two-run drive in the third inning off Chris Tillman.
The lead didn't stand up, and now the Yankees are stuck in a four-game skid and scrambling to stay in the playoff hunt.
"We've got to start winning series again," Girardi said. "We have not won the last two series and we put ourselves in a little bit of a hole."
Pineda allowed one run and two hits over five innings. After being suspended for 10 games for using a foreign substance on the mound in April, the right-hander went on the disabled list with a shoulder muscle injury. He left after throwing 67 pitches.
"We thought he started getting the ball up a little bit," Girardi said.
Baltimore played without its two starters on the left side of the infield. Third baseman Manny Machado was placed on the 15-day disabled list with a sprained right knee ligament and shortstop J.J. Hardy missed a third straight game with a sprained left thumb.
Tillman gave up two runs and five hits in seven innings.
Yankees: After throwing fastballs off flat ground, RHP Masahiro Tanaka (elbow inflammation) said through an interpreter, "I feel good because everything is progressing in the way that I want it to." Girardi said, "Our hope is he'll pitch in September."
Orioles: Machado said his injury is "not as bad as we thought." Hardy hopes to return for the weekend series in Cleveland.
Yankees: After taking Thursday off, the Yankees open a three-game series in Tampa Bay. Girardi has opted to start Hiroki Kuroda on Sunday instead of Chris Capuano.
Orioles: Baltimore gets a day off before facing the Indians to launch a nine-game road trip that includes a week's stay in Chicago, where it will face the White Sox and Cubs.
BEAST IN THE EAST
The Orioles are 30-19 against AL East foes, the best in-division record in the AL. New York is 21-25 in the division, which goes a long way toward explaining its deficit in the East.