Unfortunately, their tablet app hasn't been updated with video support yet. Additionally, you need to install the Amazon appstore app, then attempt to play a video, which will redirect you to an install link for the Amazon video app.
I really hated the Amazon appstore app because it was always running in the background and asking to update and stuff, even when I force closed it. I guess this is Amazon's way of forcing people to use their terrible app store? Even the video app itself is sluggish and feels like you are controlling a mobile site with embedded video. But whatever, at least it's possible now.
EDIT: and no chromecast support. Mega fail.