Yes, no harm at all! I don’t find it helpful to cancel the FastPass instead of letting it expire. To @Elle23‘s point, it can mess things up for your party. And unlike a restaurant, they’re not holding a specific place on the ride for you. As long as the ride is operating, the seat you would’ve used is going to be given to someone else and not wasted. It’s arbitrary whether you cancel your FP or not.
Does it really matter if your seat goes back into the FP pool and gets snapped up by someone who just happened to be browsing for a pass? Really, who is to say that someone browsing for a FP is more deserving than the poor souls waiting in a line?
That random FP might save one person 60 minutes in a standby line, but they force 2,500 people to wait a second longer. By letting your FP expire, you have just saved ~2,500 people a second of waiting. (In fuzzy math.)
tl;dr: If you end up not cancelling your FP, you’ve let the standby line move one second faster for everyone waiting. Don’t feel bad for even that one second.