I know what you are talking about as I live in North Carolina. The answer to your question is not so simple. As you said, the heat and levels of humidity make it challenging in the summer. In fact, ffor the last several days here the humidity has stayed above 70%. We just got 3" of rain yesterday. My pool is overflowing at the moment.
The source from which to pull air really is a matter of several things. Ideally, iff you have a place where you can completely contain your airspace, you can (if you have the money) set up a recirculating system that includes a propane burner that produces CO2, and a cooling system to maintain proper temps, then a good odor filtration system to scrub the air so that odors don't build up. If you can do this then you are in excellent position to not have to worry about getting outside air.
However, for small grows such as personal "closet" or "tent" grows, that level of contained, controlled atmosphere is costly, and is difficult to maintain/balance unless there is ample space for the exchanges to be set up. At that point, you have to weigh your options and look closely at what you have available versus your needs.
If you have a closet/tent grow that is only 10sqft max, you have good HVAC in your home, and you have several oxygen breathers like multiple people and/or pets present almost continuously, then you can just filter and cycle the air within your home and allow your HVAC to maintain the air temps and humidity levels, while the air breathers cycle the O2 for CO2.
If you are by yourself in a home with no pets to exchange the O2 for CO2 then you will have to pull in air from the outside. Iff you have high humidity then you have to install a dehumidifier/air conditioner within the air space that is adjacent to the grow so that the air can be modiffied before going into the grow space (at least during flowering).