What is the distinction between loving someone and being “in love” with them? Aren’t these just arbitrary constructs?

Yes, quite arbitrary.

The classical distinction is that between the concepts of Philia and Eros, the notions of brotherly love versus romantic love.

The modern distinction tends to be less sophisticated, and pretty much boils down to whether you still want to fuck somebody.