Well its obviously a bit more complicated than that e.g. the shape of the object can have a huge effect on it's falling velocity in air.

Just think of the difference between a closed and an open parachute both of which have exactly the same mass.