Eric Dier put away the final spot kick to secure victory, having been given the opportunity to put the Three Lions into the quarter-finals after Jordan Pickford saved a shot from Carlos Bacca.
Jordan Henderson was the first not to score thanks to a fine David Ospina save, fuelling fears of yet more penalties heartache for England – but then Mateus Uribe clattered the next spot kick off the underside of the crossbar.
Kevin Trippier stepped up to draw England level, before the one-handed save from Pickford and firm finish from Dier sent England through to a last-eight clash with Sweden on Saturday.
It sent fans and players alike into raptures – with millions of supporters celebrating having watched nervously in living rooms, pubs and at public screenings across the country.
The drama seemed to prove too much for defender Kyle Walker, who posted a picture on Twitter of himself stretching out on the turf as his teammates ran towards Dier and Pickford at the end of the shoot-out.
England certainly put their fans through the wringer, having been moments away from victory when Yerry Mina headed home from a corner in the third minute of injury time at the end of the second half to cancel out Harry Kane’s opener.
Dad, can you show me that photo of you guys running after reaching the quarter final?-Nah son pic.twitter.com/WbY5StQRLQ
Before the win over Colombia, England hadn’t won a major tournament shoot-out since 1996 – and never in a World Cup.
The set piece arrived after Pickford – who was criticised for his performance in the final group stage game against Belgium – was forced into a stunning fingertip save by a 30-yard piledriver from substitute Uribe.
It was the first real test of the night for the England goalkeeper, who had been a relative bystander for the majority of what was a feisty affair in Moscow.
Henderson, Harry Maguire and Raheem Sterling were all involved in clashes, and it was a spot of rough and tumble that looked like it would cost the South Americans until the injury-time drama.
In scenes reminiscent of both England penalties against Panama nine days ago, Kane grappled with Carlos Sanchez during a set piece in the 57th minute and the referee showed no hesitation in pointing to the spot.
The skipper beat Arsenal goalkeeper Ospina from the spot – his sixth strike of the tournament.
It took Kane two clear of Belgium striker Romelu Lukaku and Cristiano Ronaldo in the goal-scoring charts.
But England failed to build on taking the lead and looked a shadow of the side that played so well against Tunisia and Panama once extra time began.
Colombia looked brighter for much of the additional half hour, but Danny Rose went close with a fine effort before the clearest England chance of the night fell to Dier with five minutes left.
The midfielder – who replaced Tottenham Hotspur teammate Dele Alli in the 81st minute – missed a free header from less than 12 yards out after an out-swinging corner from the right.
Alli had struggled before going off on his return to the side and failed to make much of an effort going forward for much of normal time.
Raheem Sterling looked the most likely to get the travelling England fans off their seats in the opening stages, but the first half was a mostly cagey affair between two sides who knew a route to the final had opened up following the surprise elimination of Spain at the hands of hosts Russia.
Kane – rested against Belgium – had the first real effort on goal, connecting with a deep cross from Trippier and looping a header just over the bar from a tight angle.
There was no immediate response from Colombia, who pressed England from the front but struggled to open up a back three that had at times looked shaky against Tunisia and Panama.
Without the craft of the injured James Rodriguez – the star of the last World Cup in Brazil – they found it difficult to get the ball to striker Falcao in goal-scoring positions.
There was a moment of controversy before the referee signalled for half-time, when Wilmar Barrios appeared to headbutt Henderson in the chin as the two jostled while Trippier prepared to take a free-kick.
It seemed as though the incident had been referred to the video assistants, but the referee decided to dish out a yellow card instead.
Had Colombia gone on to win, England boss Gareth Southgate may well have felt aggrieved by the call. His side grew weary as the match wore on and the extra man could have proved decisive.
But despite a rotten history in shoot-outs, those who stepped up looked confident when the time came.
Kane, Dier and Trippier were joined by substitute strikers Marcus Rashford in hammering home with aplomb, helping secure England their first World Cup penalty shoot-out victory.
They had lost all of their previous three – against West Germany in 1990, Argentina in 1998, and Portugal in 2006.
Southgate – who famously missed the penalty that saw England eliminated from Euro 96 – will now lead his side into a quarter-final against Sweden at 3pm UK time on Saturday.
It will be the third time the two sides have met in the last five World Cups.
They went head-to-head in the group stages of the 2002 and 2006 tournaments, with the tightly fought matches ending up 1-1 and 2-2 respectively.
It’s something the players can always look back on now and draw on as an experience, Southgate said of the long-awaited shoot-out win.
We’ve talked about creating their own stories and creating their own history and we’ve done a little bit of that tonight. We go to a quarter-final, but I don’t want to go home yet.
If the trophy is to come home with him, Southgate will have to mastermind victories over Sweden – who he said England had a poor record against – and either hosts Russia or Croatia in a potential semi-final.
Get through that, and England would be in the final against France, Brazil, Uruguay or Belgium.
Brazil remain the favourites with the bookmakers, followed by France, with England third.
News Source SkyNews