Take a small building such as an Oxen Tether. Place 1 or 2 of them right outside the gatehouse(s) of your enemy's castle, but make sure their keep is enclosed first. Immediately put the Oxen Tethers to sleep. The ground troops will walk to the gatehouse to destroy them but they can't get through, so they just walk back to their spot. Your enemy's buildings outside of the castle will spontaneously crumble because the workers can't get back inside the castle. Also, the gatehouse(s) you blocked might spontaneously crumble after a while. Soon there will be no food and the peasants will leave the castle. The small army will be trapped inside their own walls awaiting your arrival. When your army is built up, you can just waltz over and wipe the whole castle off the map. (Note: Make sure there are no enemies outside of the castle, or else they will probably come and destroy your oxen tethers.