20 years ago it was much easier to give you advice. Back then most clothes makers used standard sizes. Now you have all manner of sizes claiming to be the same size. As an example (from what I have seen), XL waist ranges from 32 to 42 inches, depending on the brand.
If you plan on getting an international name such as Speedo, then you can basically do it from whatever size briefs you wear, as long as they are of the same status; ie internationally know, such as Hanes, FotL, Jokey, Calvin Klein, ect. You may have to go 1 size up on the swimsuit.
If you are planning to get a fashion swimsuit, then you need to research the brand. Most tend to claim to be size (insert measurements here), but in reality they are quite often much smaller. GoSoft tells their customers to go for the next larger size, but many seem to prefer going 2 sizes bigger.