There were only three legal buildings on Elm Avenue by September 9, 1876. Three buildings where the owners had filed permits, built properly, and operated legally. These were the Jourdan Brothers establishment at 4120 and 4122 Elm, the Elm Avenue Hotel, and Texas Jack’s Hunter's Home at 4188 Elm. All of the other buildings were cheaply made wood structures built haphazardly between these buildings and the larger hotels at the ends of the street. Just three legal buildings in an area with between 150 and 200 businesses, including hotels, ale houses, bootblacks, clothiers, booths and stalls catering to all tastes of the thousands visiting the Centennial grounds across the street.
The ramshackle appearance of many of these poorly constructed and illegally built businesses led to some calling the portion of Elm Avenue facing the Exhibition grounds at Fairmount Park "Shantytown." The illegal buildings must have concerned Texas Jack and his business partners as much as they did Philadelphia's mayor, who tried to get permission from the city council to tear down the illegal buildings for presenting a fire hazard.
Just after four p.m. on September 9, the day after the above report was printed, thousands of visitors near the entrance to the Centennial Exhibition noticed smoke rising south of them. Initial fears that the Main Building had caught fire were quickly dispelled, but as the throng of people rushed toward the fence, it became obvious that some of the wood buildings on the south side of Elm Avenue were on fire. The fire had begun at the Broadway Oyster House when a cook had grabbed a can of gasoline for the stove and carried it too close to the flame. The ensuing explosion quickly spread to encompass the surrounding buildings.
The first alarm went out from the nearby Globe Hotel at twenty minutes past four, but despite having shown their speed of response the day before by racing steam-powered fire engines down Elm Avenue, it was another twenty minutes before firefighters arrived to begin battling the blaze. When they ran their hoses to the nearest fire-plug, it was found to be clogged with sand and filth and had to be cleaned before it could be used at all. Fifteen minutes later a second alarm sounded from the United States Hotel further down Elm Avenue.
As the fire raced toward the Transcontinental Hotel to the west and the Ross House to the east, many of the one hundred thousand visitors crowding the park that Saturday gathered to watch the conflagration. The people packing the street made it hard for the firemen to move, and police were dispatched to push back the crowds. As the pine buildings burned, the flames became so hot that Exhibition fences across the street on the north side of the avenue were scorched. This heat made it impossible for the firemen to continue combating the flames. Two fire engines that had made it to the scene were kept inside the park fences, in case the winds shifted and made their way to the Main Building.
Buildings along Elm Avenue were consumed so quickly that most of the proprietors were only able to grab their cash boxes before escaping, leaving valuables and personal belongings to the flames. One zoological exhibition contained two sea lions in an enormous tank, which was far too heavy to move. Spectators watching the fire from inside the park reported hearing the animals crying above the noise. Blanchard’s Amazon Theatre, Wylie’s drinking pavilion, California Cheap John’s clothing and junk shop, and Madame Duclos’ female minstrels were all burned out, dancers of the latter being forced to flee in costume onto the Exhibition grounds. The fire’s eastward spread was halted by the solid brick walls of the Ross Building, though the interior of that building was consumed in the blaze as well.
The third alarm at a quarter to five was sounded from within the Exhibition grounds, as the fire continued to consume buildings along Elm Avenue. A bowling alley, a shoe shop, more beer saloons, restaurants, and a stable were consumed before firefighters were finally able to contain the flames. The following morning’s newspapers, in addition to recounting the story of the fire, pondered its origin. Arson, either on the part of the oyster house staff or from a city official determined to shut down the less-family-friendly establishments contained in the area, were the primary guesses as to the beginnings of the blaze.
The mayor wasted no time in sending crews to tear down all of the remaining buildings, legal or otherwise, including Texas Jack's Hunter's Home hotel. Jack, far to the northwest chasing Crazy Horse in the aftermath of the Little Bighorn battle, wouldn't learn of the fate of his hotel and his fortune for weeks. By then there was nothing left of either to salvage, and the trajectory of Jack's life changed. He had been convinced that this was his ticket out of constant touring, to starting a family and spending time in leisure with his wife. None of that was now possible, so after a frustrating campaign with General Terry, Jack headed west on a trek with Sir John Rae Reid and planned his next dramatic tour, his first without his friend Buffalo Bill.