Jamie Mills came off the bench to grab a dramatic last-minute goal as Dave Mackay enjoyed a winning start as Albion manager.
The former Rangers full-back, whose last appearance was against Annan on the opening day, pounced from close range to net his first senior goal at the end of a pulsating second half.
Elgin had won 4-0 at Forthbank in September to end Stuart McLaren’s reign as manager but Albion gained revenge against Jim Weir’s men, at a venue where they were beaten three times last season.
Ross Forsyth, Ross McGeachie, Darren Smith and Isaac Layne were all ruled out through injury so there was a recall for Blair Henderson while our former midfielder Willie Robertson slotted in at right-back as a trialist following his recent departure from Annan.
Clear-cut chances were at a premium in the first half but Chris Smith saved well from Chris McLeish before Elgin took the lead in 29 minutes when defender Darryl McHardy fired in a low shot after we failed to clear our lines.
Albion struggled to create openings before the break but it was a different story after the interval and it took only 20 seconds for top scorer Marc McKenzie to net the equaliser, heading an Eddie Ferns cross past Marc Waters at the back post for his seventh goal of the season.
Henderson put Stirling in front just before the hour when he followed up to score after Waters had pushed out a Sean Dickson shot, only for Brian Cameron to haul City level after a corner was only partially cleared.
Both teams went for it in the closing stages and Mills proved the unlikely hero when he smacked the ball home, after Waters had parried a header, to reward a huge team effort with all three points.
Elgin City: Waters, Cooper, MacPhee, McHardy, Nicolson, Reilly, Reid (Macleod 72), Cameron, McLeish (Gunn 62), Sutherland, Dodd (Moore 62).
Subs (not used): Stewart, Bruce, Cullen, Watson (GK).
Goals: McHardy 29, Cameron 76
Stirling Albion: C Smith, Robertson (Verlaque 87), Petrie, McMillan, Scott Davidson, Hodge, Beith, McKenzie (Mills 87), Dickson, Ferns, Henderson (Kavanagh 75).
Subs (not used): R Smith, Docherty, Morrison, Binnie (GK).
Goals: McKenzie 46, Henderson 59, Mills 89
Referee: Scott Millar