The song's lyrics were inspired by the band's first and ill-fated shows in Halifax, Nova Scotia (a major city on Canada's east coast) in 1990; "Hello City" is a nickname for Halifax.
According to Steve in Public Stunts, Private Stories:
"...we played at this bar called The Lower Deck, where they had mostly Celtic music, being as there are mostly sailors in there, drinking. [...] We stayed at this place called The Carlton Hotel, on Barrington Street, which was upstairs from this metal bar called Rosa's. It's no longer there, but it was a total flophouse. [...] The room that Ed and I shared had four beds. One of them was up against the wall and another had a mattress falling through its frame. There was a bathtub with no shower. It was just gross. [...] Halifax on a weekend, you can't go anywhere without seeing somebody bleeding, somewhere. To get to the street from our terrible room, you had to walk down three or four flights of stairs and then step over some bleeding body. We were young, none of us were real drinkers, Ed didn't drink at all, and the whole drink culture of Halifax was just shocking to us. [...] So we were playing at The Lower Deck, this sailor's watering hole, and no one would pay attention. I remember taking a picture from the stage and when I got the pictures back, there was nobody looking at the camera. [...] I remember thinking I hate this place, I hate this place. [...] We've actually had great times there since, but that's because we learned the "when in Rome" axiom. You know the expression, 'do at the Romans do?' So 'when in Halifax,' we learned to 'do as the Haligonians do,' which in this case means drinking and bleeding." 
The song was first recorded for the unreleased demo tape Barenaked Recess in 1990; before being re-recorded for the 1991 promo CD Variety Recordings, the band's full-length 1992 debut Gordon, and their 1996 live album Rock Spectacle.
- Towards the end of the song, Steve quotes the chorus of English rock band The Housemartins' hit 1986 song "Happy Hour", which, apart from similar subject matter, is also in the same key and has a similar chord progression.
- Public Stunts, Private Stories by Paul Myers - pages 48-49