Sikandar Raza proved the batting hero for the hosts as his unbeaten 60 saw him shepherd the lower order to their 220 run target with six balls to spare.
Spinners George Dockrell (2-29) and Andy McBrine (2-52) had helped reduce Zimbabwe to 171 for 7, and at that stage Ireland had edged ahead.
However Raza held his nerve, finding a suitable ally in debutant Wellington Mazakadza (10) as they shared in a vital 8th wicket stand of 38.
Former Lisburn and Northern Knights batsman Craig Ervine was the other player to score a half century in the Zimbawe innings - he struck five boundaries in a confident 60 from 78 balls, before becoming McBrine's second victim.
George Dockrell was the best Irish bowler on view, removing the dangerous Sean Williams (24) and skipper Elton Chigumbura (16), but couldn't dislodge the stubborn Raza.
Earlier Gary Wilson top scored with an unbeaten 70 as the Irish recovered from a middle innings wobble to post 219 for 8.
Wilson shared stands of 83 for the 4th wicket with Ed Joyce (53) and an unbroken 43 in just 28 balls with Andy McBrine (23*) for the ninth.
Joyce and Wilson rebuilt the Irish innings after Paul Stirling (24), William Porterfield (9) and Andrew Balbirnie (5) fell in quick succession.
Their repair job saw the Irish reach 137 for 3 with 15 overs left and with a solid platform, hopes of a total of 250 looked feasible.
However Joyce's dismissal saw the Irish lose five wickets for 39 runs - one of which was Kevin O'Brien (16) playing in his 250th match - and at 176 for 8, they were in deep trouble before Wilson and McBrine's counter-attack.
Zimbabwe leapfrog Ireland in 10th place in the ICC rankings table, but the Irish have a chance of instant revenge when the teams meet again at the same venue on Sunday before the series concludes on Tuesday.
Ireland v Zimbabwe, 1st ODI,
At Harare Sports Club,
IRELAND 219/8 (50 overs, G Wilson 70*, E Joyce 53, P Stirling 24. A McBrine 23*, J Nyumbu 2-35, W Mazakadza 2-45)
ZIMBABWE 222/8 (49 overs, S Raza 60*, C Ervine 60, S Williams 24, C Chibhabwa 22, G Dockrell 2-29, A McBrine 2-52)
Zimbabwe beat Ireland by two wickets
