An improved test rig providing both the heat and cold source was used to perform thermal response test (TRT), and the line source model was used for data analysis. The principle of determining the temperature difference between the inlet and outlet of test well can keep the heating or cooling rate constant, along with a reduced size of test rig. Among the influencial factors of the line source model, the temperature difference was determined as the most important, which agreed with the test results. When the gravel was taken as the backfill material, the soil thermal conductivities of heating and cooling at the test place were 1.883 W/(m·K) and 1.754 W/(m·K), respectively, and the deviation of TRT between heating and cooling soil was 6.8%. In the case of fine sand, the thermal conductivities of heating and cooling were 1.541 W/(m·K) and 1.486 W/(m·K), respectively, and the corresponding deviation was 6%. It was also concluded that different velocities of water had less influence on TRT than the temperature difference.