J.J. Hoover (0-5) walked Allen Craig to lead off the 10th. One out later, David Freese singled Craig to second, extending his career-best hitting streak to 19 games, tied with San Francisco's Marco Scutaro for the longest in the National League this season.
Daniel Descalso followed with his RBI double off the left field wall. Matt Carpenter added two insurance runs with a bases-loaded single before Holliday hit his homer -- a 464-foot shot into the left field seats off Curtis Partch, who was making his major league debut. The homer was Holliday's 10th of the season.
Trevor Rosenthal (1-0) had four strikeouts in two perfect innings to get the win. The victory meant that the Cardinals have now either won or split 13 straight series.
The Reds grabbed a quick 2-0 lead in the first. Shin-Soo Choo led off with an opposite-field double into the left-field corner and moved to third on Derrick Robinson's sacrifice bunt. After Joey Votto walked, Brandon Phillips nudged a half-swing bloop single into short right field to drive in Choo and send Votto to third. Votto scored on Jay Bruce's sacrifice fly.
St. Louis tied it in the fourth. Carlos Beltran led off with his 14th home run of the season, a 409-foot blast to center field on a 3-1 pitch. Holliday followed with a double and scored one out later on Yadier Molina's opposite-field double into the right-field corner.
A two-run fifth gave the Reds a 4-2 lead. Choo led off with a single to right and Robinson was nicked by a pitch while squaring to bunt. Votto struck out and Choo moved to third on Phillips' fly ball to center field before Bruce lined a two-run double into the right-field corner.
The Cardinals jumped all over reliever Sam LeCure to tie the score again in the seventh. Jon Jay led off with a single, his third hit of the game, and went to third on pinch-hitter Matt Adams's double to center that Choo got a glove on but couldn't hold. Jay scored and pinch-runner Shane Robinson went to third on Carpenter's single. Beltran greeted reliever Alfredo Simon with a game-tying sacrifice fly.
Carpenter extended his career-high hitting streak to 18 games.