I originally had breast cancer 18 years ago. At that time I had a very small tumor with no mets so I had a lumpectomy, radiation, and then 5 years of tamoxifen. I didn't have chemo. I had no side effects effects from the tamoxifen at all.
I currently have IBC in the same breast as 18 years ago, but all of my doctors have said it is not a recurrence but a totally new cancer. I am currently in chemo this time, and while some people do have some side effects from tamoxifen, I can't even imagine in being anywhere near as bad as the side effects from chemo. I think you are over the hump and feeling normal again may still take some time, but that would more likely be from the lingering effects of the chemo rather than tamoxifen.