Work in physics has a different meaning than work in everyday life. In physics, it is specifically only means the moving of an object in a particular direction.
Work = Force * Displacement