Harry Kane scored just after the break to hand the Three Lions a deserved lead but Yerry Mina's stoppage-time header sent the game to extra-time and, eventually, penalties after neither team could find a victor in open play.
The win sees England move on to the quarters where it will face Sweden, while Colombia goes home. But, the one name that Jose Pekerman had hoped to be fit is struggling to recover in time.
Davinson Sanchez has played day-in, day-out against all of the players likely to be in the England starting eleven.
Will the Three Lions win a first knock-out match for 12 years?
"We are a better team than four years ago", explained goalkeeper David Ospina to reporters, adding that the experience and unity in the current squad gives them a great chance of prevailing. Though Colombia are risky without Rodriguez, having their star available would be a huge boost, as the Bayern Munich man was involved in 48 percent of Colombia's goals in qualifying, and added two assists in just 152 minutes so far in Russian Federation.
It was England's first win in a knockout game since 2006 and earns them a quarter-final with Sweden, who were far from impressive in beating Switzerland 1-0 earlier on Tuesday.More news: Hawks-Mavs swapping Doncic and Young
It is the furthest England has progressed in any tournament since the David Beckham era when a golden generation of players underperformed on the worldwide stage, exiting the 2002 and 2006 World Cups in the last eight. Even England's 13.3 shots per game, eighth-best in the group stage, is skewed by piling up 18 shots against Tunisia.
At full-time, Colombia had committed significantly more fouls - 21 to to England's 13 - while England marginally stayed on top of possession, 52% to 48%.
The South Americans had been getting away with blatant holding and wrestling at each of England's many corners and the referee's patience finally snapped when Carlos Sanchez hauled down tournament leading scorer Kane once too often.
Fox (live in most markets) and Telemundo are both included in the "Fubo Premier" bundle, which has a channel package that is largely tailored towards worldwide soccer fans and also has World Cup live streams available in 4K. He told the London Evening Standard: "We are all excited about the chance to end this run without a knockout victory".
England captain Harry Kane had given his side a 1-0 lead in the 57th minute with a well-taken penalty. Bet on England to win in a close one.
"But when we are in possession, it allows me to get further forward to get beyond the striker a bit more and get into goalscoring positions", he said.