Downloading is absolutely illegal, it’s just not really enforced because you need to prove criminal intent. You’re still accessing copyrighted material without a license, which is a copyright violation.

Distribution has much higher penalties and is more likely to push people to buying (harder to find copies = potentially more legal sales), so that’s where enforcement is focused.