I think you're right to be concerned. Chemo has a load of side effects, and if it's not needed, then why do it? That said, it sounds like maybe your current healthcare team thinks you need it. So, the question is, how do you figure out the best treatment plan for you?
How is it that they know it's a secondary cancer due to radiation?
What is the risk that the cancer may recur without chemotheraphy?
Did you have a biopsy? What were the results?
And loads of other questions....
I think the need for any treatment plan needs to be clearly explained to every patient.
Cancer treatment, unfortunately, has a lot of uncertainty. So, it's very possible that chemo is NOT clearly indicated but that the doctor thinks it's a good idea... Definitely, it sounds like you need another appointment to make sure you have all the information you need to make a strong decision.
Lastly, I would suggest the possibility of asking for a second opinion.
That way, you'll have two professional opinions to work from.
Good luck - I'm sorry you have breast cancer.