There can be different factors. Texture size doesn't really matter. Number of vertices(polys) can be a factor with slow graphics cards. Other things that can make a difference a number of separate textures and number of separate meshes per car/track, the more the lower the fps. But then meshes shouldn't be too large too.
Could you run your test with
Code:
vdrift.exe -profiling -dumpfps
And post the log.txt from C:\Users\<user>\Documents\VDrift
Code:
INFO: Loading track from path: data/tracks/ruudskogen
INFO: Loaded surfaces file, 6 surfaces.
INFO: Track timing sectors: 2
ERROR: Failed to load "objects.txt" from:
C:\Users\nan\Documents\VDrift/tracks/ruudskogen/objects
data/tracks/ruudskogen/objects
data/carparts
data/trackparts
INFO: Car loading was successful: XS
INFO: Current FPS: 36
INFO: Current FPS: 43.8
INFO: Current FPS: 72
INFO: Current FPS: 46.4
INFO: Current FPS: 71.1
INFO: Current FPS: 47.6
INFO: Current FPS: 39.7
INFO: Current FPS: 43.6
INFO: Current FPS: 39.7
INFO: Current FPS: 61.2
INFO: Current FPS: 67
INFO: Current FPS: 69
INFO: Got quit message from GUI. Shutting down...
INFO: Profiling summary:
ai: 0.0115627 %
car: 0.758215 %
physics: 3.10223 %
render: 76.3541 %
scenegraph: 0.331627 %
sound: 0.0412931 %
INFO: Shutting down...
INFO: Exiting