I had an 04 Tacoma with the 3.4 liter. The towing capacity was 6000lbs also. I have a travel trailer with a GVWR of 5,900 and I'd estimate the weight was actually about 4,500-5,000 the way I had it loaded. The truck did horrible. It was very stressful to tow the trailer and the automatic transmission spent all its time shifting between 1st and 2nd gear on hills. It was very slow going. I took it through the Rockies one time and a drive that should have been 5 hrs took 9. Not to mention I got 6-10 miles per gallon depending on the terrain.
It is possible that the 4.0 would do better. Also I did have a 3 inch lift with slightly bigger tires. A regear might have made it better. I ended up selling the Tacoma and getting a Dodge diesel. I wish I had kept the Tacoma and got a different trailer. Hope that helps.