The cheapest horses you can buy are paint horses and you can purchase them in bruma. They
are , however , slower and weaker than all other horses as well. You can get a free one
during the first quest of the main questline at weynon priory. You also get the very best
horse , faster and immortal , during the dark brotherhood questline. If i were you , i'd
wait till i'd get her and not waste septims on other horses , unless you are not planning
on completing the DB questline.

the two cheapest horses are chestnut and paint but DO NOT BOTHER, when you go and give
jauffre the amulet ask prior mable "asstince" and he will give you his paint horse for
free, plus in dark brotherhood you get an invincible, fast horse, as a matter of fast it
is THE fastest in the game but yeah anyway the chestnut costs 1k i bieleve and the paint
costs eaither 400 or 600