We did it back in December (the 19th) when the park was pretty packed (Neil Patrick Harris was narrating Candlelight Processional that night). We had FastPasses for IllumiNations, so we didn’t have as far of a walk as those who were watching from the other side of the lagoon, but it was pretty much a mass of humanity heading through Future World and out of the park. There was a crowd for the monorail, but it didn’t seem all that horrible. If we didn’t make the first train, I’m pretty sure we made the second one. We were going to the Grand instead of the Poly, but same difference. The only upside to the Poly is that, as another poster mentioned, depending on where your room is located, you could skip the switch to the resort monorail at TTC and just walk to the Poly.
We didn’t find it bad at all, but we hadn’t been up since 4:30 that morning either. You’ll definitely be running on fumes by the time you get to IllumiNations, much less the commute back to the Poly. In your case, I’d be tempted to do as someone else mentioned, watch from a spot closer to the International Gateway, head to Beach Club after IllumiNations, and arrange for Uber/taxi/Minnie Van/Whatever to take you “home”. It’d probably be well worth the cost.