I used to have a somewhat large workshop in an old barn with a dirt floor, and while the problem was merely inconvenience, and nothing show-stopping, it does eventually get to you. It is true that as long as you take care of your tools, rust won’t win any battles, and as long as you find ways to deal with the uneven surface, you’ll be OK.
Having said that, sometimes you’ll be doing (or perhaps better worded as “attempting to do something”) which requires your very best, and if, in those moments, a particularly problematic portion of floor rears its ugly head, it can tip the scales in your ability to have optimal results.
Also, sometimes you just get plain sick and tired of dealing with the uneven floor at every turn, and that can lead to momentary frustration.
I would look into any avenue available to you to create even just a small section of level, comfortable floor. That’ll be your primary standing / working area. Maybe you just pour some gravel, level that, put brick or etc over the top, and tamp it all down. Maybe you can install a wooden floor. But even just a few dozen square feet of space might be enough to give you some stable footing for the important tasks.