NEW YORK -- The implications of Nick Swisher's tongue-lashing didn't become clear until the ninth inning on Wednesday, when the right fielder's spot came up in the lineup with a man on first base.

Since Swisher had been ejected for arguing balls and strikes two innings earlier, Brett Gardner walked to the plate in his place. And he did his job, laying down a bunt to move Robinson Cano into scoring position.

Yet it's hard not to consider how things might have turned out had Swisher, the Yankees' leading home run hitter, had a chance to bat.

"I was a little too upset at the time to even think about that," Swisher said after the Yankees' 4-3 loss to the Rays.

Swisher was almost immediately ejected in the seventh inning after a called third strike brushed the outside corner of the plate, making him the second Yankee to be tossed in three days.

Manager Joe Girardi had been thrown out of Monday's game against the Red Sox for arguing a called third strike to Derek Jeter.

"There was no malicious intent," Swisher said. "It's not like I was trying to completely show him up. I just think my emotions got the best of me."

In the ninth inning, Rays outfielder B.J. Upton and manager Joe Maddon argued a called third strike for several moments with home-plate umpire James Hoye, though neither was ejected.