Where are you going to use it? Is portability the most important aspect? As in, do you want something smaller than your phone?
Bluetooth or wired? Are you just connecting Bluetooth headphones/earphones, or are you connecting wired headphones. If the latter, do your headphones need a lot of power (typically bigger, higher-end headphones).
Do you want to stream music as well? Smaller, less expensive MP3 players typically have no streaming abilities. Look for something that runs on the Android operating system. This will let you install most apps like Spotify, Qobuz, Tidal, and others.
How much do you want to spend? There are lots of great options under $100, and you can spend over $1,000 for high-end audiophile models. Some of our favorite models are under $250.