Preheat oven to 375°F. Butter or spray a 9×13-inch baking dish. Peel and slice potatoes ⅛ inch thick using a mandoline or sharp knife. Do not rinse the slices. Thinly slice the onion.
Make the cream sauce: Melt butter in a medium saucepan over medium heat. Add garlic and cook for 30 seconds. Whisk in flour and cook, stirring constantly, for 1 minute. Pour in warm milk gradually, whisking to prevent lumps. Add broth, thyme, nutmeg, salt, and pepper. Cook over medium heat, stirring often, until the sauce coats the back of a spoon, about 4–5 minutes. Do not let it boil.
Layer half the potato slices in the baking dish. Scatter half the onion over the top. Season with half the divided salt and pepper. Pour half the cream sauce over and spread to the edges. Sprinkle with ¾ cup cheese. Repeat layers: remaining potatoes, onion, seasoning, sauce, and top with remaining ¾ cup cheese.
Cover tightly with aluminum foil. Bake for 45 minutes. Remove foil and bake for 40–45 more minutes, until deeply golden and bubbling. Test doneness by inserting a thin knife in the center — it should slide through with zero resistance. If there's any drag, cover and bake 10 more minutes, then test again.
Rest uncovered for 15 minutes before serving.