Scotland qualified for their first major tournament since 1998 after David Marshall saved Aleksandar Mitrovic's penalty in a shootout to send them to Euro 2020.
Scotland put their fans through the wringer as Steve Clarke's men were two minutes away from a 1-0 win in normal time in the winner-take-all clash after Ryan Christie's goal but a Luka Jovic injury-time equaliser forced extra-time.
Clarke had taken off his three most influential players - Lyndon Dykes, John McGinn and Christie - just before Serbia's dramatic equaliser and Scotland lost their spark in extra-time but they held on to take the game to penalties.
In the shootout, all five of Scotland's takers found the net before Mitrovic was denied by Marshall to send the Tartan Army through in dramatic fashion.
Scotland's reward for the win in Belgrade is a place in Group D alongside England, Croatia and Czech Republic, and a mouth-watering Euro 2020 schedule that will see two of their matches - against Croatia and Czech Republic - played in Glasgow.
How Scotland did it the hard way...
Scotland have been turned upside down by Clarke's influence in the dugout with a run of eight matches unbeaten and he stuck with the crux of the team that beat Israel on penalties in the semi-final.
Despite being the away team, Scotland enjoyed 77 per cent of possession in the opening 15 minutes with Dykes dominating the Serbian back three which gave the visitors the platform to build on.
Serbia were barely seen as an attacking force but flexed their muscles midway through the half when Mitrovic laid the ball back for Sasa Lukic who steered the ball a yard wide from the edge of the box.
Scotland started to turn possession into chances when Christie fed McGinn after another knockdown from Dykes but the in-form Aston Villa man struck his effort too close to the goalkeeper.
Clarke perhaps sensed an opportunity to take advantage of Serbia's lacklustre showing after the break and skipper Andy Robertson spurned a glorious chance five minutes after the break after Dykes again had provided the brilliant build-up play.
But they only had to wait 90 seconds to get the goal their play deserved.
Callum McGregor snapped into an interception and fed Christie, who swivelled into space and found a cute finish into the corner from 22 yards.
Christie soon had a volley saved following a Dykes head-on and Scotland remained in control.
But the second goal did not come and minute by minute Clarke's side slipped deeper into their own box. Headers from Sergej Milinkovic-Savic and Mitrovic failed to hit the target but Scotland did not look overly nervous as they continued to push on.
McGregor saw a long-range effort swerve just wide and Christie curled another clever effort just wide.
Clarke took off his three most creative players in order to see the game out, but the pressure increased and Jovic gave a sighter of what was to come by heading wide when free inside the box.
He did not waste his second chance though as heartbreak was on the menu again for Scotland.
Jovic evaded marker Scott McTominay amid a crowd of bodies to head home unchallenged from Filip Mladenovic's corner to deny Scotland and force extra-time.
It was one-way traffic in the extra period as Marshall was called upon to produce a brilliant fingertip save to divert Nemanja Gudelj's long-range effort away from the top corner.
But Scotland held out for penalties and Leigh Griffiths' first involvement from the bench to was to fire home the opening penalty before McGregor, McTominay, Oli McBurnie and Kenny McLean also netted before Marshall guessed the right way to deny Mitrovic.
The goalkeeper stopped for a second to check with the referee that he was on line before the team celebrated an achievement that is long overdue for Scottish football.
Man of the match: Ryan Christie
Marshall pushed him very close by stepping up at the big moments in the match but Christie was the best player on the pitch. He roamed free in behind the impressive Dykes all night and linked the play between the lines with great maturity and trickery. This was on show for what looked to be his match-winning moment - a goal that would have graced any major tournament. A player of his talent deserves to be playing on such a stage.
Christie told Sky Sports: "It is just an amazing night. From the start, we kind of believed. Obviously, the last few camps we have picked up so much belief in each other. Even the way the game went tonight, conceding that late equaliser and still digging in. Penalties and you're away from home but big Marshy again comes up. He's unbelievable.
"When you are on the pitch, you always feel like you can influence it. When you are off, you can't watch. Those penalties are the worst thing I have ever been through. It's just everything."
What the manager said
Scotland head coach Steve Clarke told Sky Sports: "Normally, we fall at the final hurdle or we have the glorious failure, but I thought we were the better team in the game and deserved to be in front. We just couldn't get the second goal to kill them off.
"To concede then from a set play when they were throwing everybody into the box was a big setback, but you have to be so proud of the players, the way they responded to that.
"That was a sore one as we were really close to where we wanted to be. The reaction was to dig in during extra time, and we managed to do that. We got to penalties and we've managed to produce again from the penalty spot. It's a magnificent night for everyone in Scotland.
"We said we'd try to put smiles back on faces and I hope they managed to enjoy the match as we did here in Belgrade. They can smile tomorrow."
'Mitrovic was thinking about glory and headlines'
Darren Fletcher on Sky Sports:
"Unbelievable! The shoot-out was excruciating! I am buzzing! David Marshall is a hero to the whole country! But the standards of all the penalties was brilliant.
"Talk about character. Those lads weren't stepping up thinking, 'Imagine if I'm the one to miss'. They were positive - and then Marshall was the hero.
"I fancied every one of them stepping up. There was something about Mitrovic; he was saving himself for fifth. Our main striker stepped up first - bang, into the back of the net. People thinking about glory and headlines; that's what I thought about Mitrovic.
"Then Big 'Marsh' - what a hand."
Opta stats
- Ryan Christie's opener was Scotland's first ever goal against Serbia, in their third meeting in all competitions.
- Scotland are unbeaten in nine consecutive international matches in all competitions (W6 D3), last enjoying a longer run without defeat in February 1930 (11 games).
- Serbia have now failed to qualify for the European championships in each of their four attempts since first competing as an independent country in 2006.
- Christie has scored four goals in his last five games for Scotland, having failed to net in his previous nine for his country.
What's next?
Scotland are back in Nations League action on Sunday with a trip to Slovakia before taking on Israel on Wednesday.