Second-half goals from Scott Robinson and Lyndon Dykes saw Livingston stun 10-man Celtic 2-0.
Celtic's unbeaten start was ended by a Livingston side who outfought and outplayed their visitors - and Neil Lennon's side were later leapfrogged at the Scottish Premiership summit by Rangers after their 5-0 win over Hamilton.
The Bhoys' task here was not aided by Ryan Christie's needless first-half red card for a reckless high challenge on Robinson, which left him needing treatment.
Robinson would have his revenge on Lennon's side after the break, beating Fraser Forster with a fine finish two minutes into the second half after an excellent Robbie Crawford pass.
And the hosts doubled their lead in style when Dykes latched onto a long ball from goalkeeper Matija Sarkic before lifting the ball over Forster, before comfortably holding firm to secure their first win over Celtic at the 24th attempt.
How Livingston stunned the leaders
Livingston avoided defeat against Celtic in both of their previous two league games, but with the Bhoys entering the game on the back of an impressive 2-0 win over Cluj in the Europa League on Thursday night, there was no hint of an upset coming into Sunday's game.
From kick-off, though, the hosts had more fight and more willingness to battle for every ball, and only their final pass stopped them turning a number of early half-chances into something more threatening.
At the other end, Celtic looked dangerous when they occasionally made forays forward, but they were dealt a blow before the half-hour when Christie lost the ball and in his frustration, flew in high on Robinson and was given his marching orders.
Livingston finally made their build-up play count within two minutes of the game restarting after half-time as Crawford was given acres of space to turn and spot Robinson's run between Kristoffer Ajer and Christopher Jullien, and he opened the scoring with a shot high to Forster's left.
Celtic showed little sign of improvement, still failing to test Sarkic enough to ever worry the home side, and he ended up providing the assist for Livingston's second which would ultimately end the game as a contest.
A long ball downfield was allowed to bounce by a stagnant away defence and Dykes was only too happy to latch onto it, and coolly lobbed Forster, who was stuck in no-man's land, to double the lead.
Lennon hauled off captain Scott Brown for Olivier Ntcham as he looked to throw the kitchen sink at the hosts - but an afternoon of toil ended as it had begun, with Livingston holding firm to earn a well-deserved win.
What the managers said
Livingston boss Gary Holt: "The performances have been really good, last week was a sore one but this was the same level of performance. It comes from big Lyndon putting his shift in, and everyone feeds off that. I thought we thoroughly deserved it.
"I thought we were better against 11 men. We got a little bit excited because it's 10 men, but they're an exceptional side. You can't leave gaps, you've got to be patient. The first goal is a great goal, I'm delighted for Robbo."
Celtic manager Neil Lennon: "The red card is a major turning point in the game, I've no issue with it, it's late and it's high. Apart from the two goals, we were a little bit disappointed with the second goal, because we were well in the game at 1-0 and you have to throw everything at it, but we didn't do the dirty bits of the game well enough with 10 men.
"You always worry after a big European night, the come down, and coming to a place like this is always difficult."
What's next?
Livingston are away to Kilmarnock on Saturday, October 19 at 3pm, while Celtic host Ross County at the same time.