Meanwhile, Tonga held out a tough Lebanon to progress to their semi-final after managing a 24-22 victory.
It is the first time a tier two nation has beaten a tier one nation in a World Cup quarter-final making for arguably the greatest win in the event's history.
Shaun Johnson levelled before Taane Milne scored the decisive points with another penalty after 62 minutes.
As it was, the two points from hooker Koroisau's successful 15th minute effort were all that divided the teams.
Captain Kevin Naiqama was reduced to tears after Fiji stunned New Zealand to set up a World Cup semi-final against Australia.
Fiji started the match with an intensity the Kiwis simply couldn't match, resulting in New Zealand giving away a number of early ruck penalties in attempting to slow the play the ball down, one of which Fiji used to take an initial 2-0 lead.
GETTY IMAGES Fiji five-eighth Jarryd Hayne didn't take on the Kiwis' defence too frequently
He said: "It's indescribable".
"I am so proud of these boys".
"I didn't get any sleep last night and really thought about what I want to do, what I want to achieve". I never stopped believing.
"We slayed a giant tonight", he said.
"We laid the foundations in 2008 and we have seen the fruits in 2017".
He says this is one of the many reasons we don't want people trying to get onto the pitch- because it's risky. "We will be preparing well and we know what's coming, we have got a big week ahead of us".