Dear Urban Diplomat: Is it rude to order dessert when there’s a line of people waiting for a table?

Dear Urban Diplomat, My husband and I recently waited for two hours to get a table at a new restaurant at Dundas and Bathurst. While we ate, we could feel the people in line eyeing our table. After we finished our entrées, the server brought our bill without offering dessert, so we sent it back and ordered panna cotta and cappuccinos. As we did, someone in line let out an exasperated “C’mon!” which I found incredibly rude. But then I wondered if maybe we had violated some unwritten rule. Is it bad form to order dessert when there’s a big lineup? —Just Desserts, Dufferin Grove