db:migrate:reset should not drop all tables

db:migrate:reset is definitely useful as well.
but I'd like something that would just revert all migration calling all the down() methods one by one from the last to the first.

If I understand correctly, you're trying to roll back every migration my calling down() (like Mathieu said) On 2.3.5, rake db:migrate VERSION=0 just seems to work:

~/rails$ rails -v
Rails 2.3.5
~/rails$ rails migrate_reset
      ...
~/rails$ cd migrate_reset/
~/rails/migrate_reset$ script/generate model user name:string
      ...
~/rails/migrate_reset$ script/generate model book title:string
      ...
~/rails/migrate_reset$ rake db:migrate
(in /Users/jeff/rails/migrate_reset)
==  CreateUsers: migrating ====================================================
-- create_table(:users)
   -> 0.0015s
==  CreateUsers: migrated (0.0017s) ===========================================

==  CreateBooks: migrating ====================================================
-- create_table(:books)
   -> 0.0012s
==  CreateBooks: migrated (0.0013s) ===========================================

~/rails/migrate_reset$ rake db:migrate VERSION=0
(in /Users/jeff/rails/migrate_reset)
==  CreateBooks: reverting ====================================================
-- drop_table(:books)
   -> 0.0026s
==  CreateBooks: reverted (0.0027s) ===========================================

==  CreateUsers: reverting ====================================================
-- drop_table(:users)
   -> 0.0019s
==  CreateUsers: reverted (0.0021s) ===========================================

~/rails/migrate_reset$

I've also tried this on edge. Same results. I think this one can be marked as invalid.