Monooctadecyl maleate, as a polymerizable surfactant, was synthesized by the mono-esterification of maleic anhydride and octadecanol, and was utilized to surface-modify nano-Fe3O4 particles. A polymerizable magnetic fluid was obtained by directly dispersing modified nano-Fe3O4 particles into styrene monomer, and the polystyrene/nano-Fe3O4 composite was prepared through free radical polymerization of polymerizable magnetic fluid. The structure and dispersion status in different dispersion phases of modified nano-Fe3O4 particles were studied by Fourier transform infrared (FTIR) spectrometry, X-ray diffraction (XRD) and transmission electron microscopy (TEM), respectively. The experimental results show that the nano-Fe3O4 particles modified by monooctadecyl maleate with the size of about 7-10 nm can be uniformly dispersed into styrene and fixed in the composite during the procedure of polymerization. Thermogravimetric analysis (TGA) and vibrating sample magnetometry (VSM) indicate that the thermal stability of polystyrene/nano-Fe3O4 composite is improved compared to that of pure polystyrene, and the composite is a sort of superparamagnetic materials.