Wests Tigers will this week celebrate the 50th anniversary of Western Suburbs' famous 8-3 loss to the mighty St George in the 1963 Grand Final at the SCG.
Controversy surrounding the referring performance of Darcy Lawler persists to this day, with Kelly and others still convinced Lawler had a wager on a St George victory.
"Having been there, having seen it all take place, it's pretty hard to forget," said Kelly.
"We were never going to win, simple as that."
An apparent try for Wests was ruled out and Johnny King scored a try for St George after appearing to be tackled by two Wests players.
"He's been told to get up," said Kelly. "The referee told him to get up and he was telling us to get off him.
"No I haven't forgiven him, how could I?
"We were all struggling, battling to work and play. We were getting no money to play, getting belted every weekend.
"To see yourself in a position to win If Wests were to have won that Grand Final all that images of Ian Walsh and blokes like that holding up the trophy, that would have been us.
"That would have been Arthur Summons standing up there."
Kelly and the other stars from that famous Grand Final became part of arguably the greatest Kangaroo side to ever go on tour, claiming the Ashes in England for the first time following a 50-12 destruction of the previously all conquering British.
Wests and St George players made up a large part of that touring side and the friendships formed in the earlier 1960s are testament to power of Rugby League ability to bring people together.
"The 60's were the greatest days, I'd do it all again," said Kelly.
"We're all great mates. Our wives are too. When we'd play them at the SCG, all our wives would sit together.
"I still go out and see Reg Gasnier and the others."
3pm on Sunday will be a special time for Kelly and his mates as the two sides make their way on to the SCG they've graced so many times before.
"It just makes you feel good to walk out there," said Kelly.
"I enjoy watching the young players get their chance. They've got plenty of guts.
"We should never have left the SCG."