"The setting you want to use should be a little different than for an ipod, for best picture. I would use 24 fps "frame rate", video bit rate 1200ps, audio bitrate 320kbps, audio sample rate 48hz and resolution 240 x 432 deinterlaced...Now according to your movies/videos you can adjust settings as needed."
I converted the video file using those settings with the program mentioned. When I go to play it on my Instinct, it won't load.
Nickname_sin has one of the best, detailed writeup for coverting DVD's and getting them on your Instinct. Here is the link...
http://www.instinct-samsung.com/index.php/topic,2644.0.html