When sneaking around at night, you need to get by many of the prefects. When you need to get by them, simply crawl under the bookcase to the left of the portrait that leads to the Griffondore common room. It leads through a secret passage that takes you to the fourth floor.