Many people seem to be confused about the TempData feature in ASP.NET MVC. TempData behaves like the ViewData dictionary, except that it persists until the next request from the same browser.
Because of this, TempData should only be used when you know exactly what the next request is going to be. If you set TempData and your action then returns a ViewResult, then the next request, whatever it happens to be (an AJAX request, another page the user opened in a different tab, etc.), is going to see the TempData value you set, and no other request will see it. I'm guessing that this is never the behavior you want.
Realistically, this means that you should only use TempData immediately before you redirect (in other words, immediately before you return a RedirectResult), because this is the only time when you are sure what the next request from that browser is going to be. I actually think that RedirectData would be a better name for this property. Someone, perhaps Steve McConnell, once noted that you should never include "temp" in a variable name, since variables are, by definition, temporary. "Temp data," to me, is synonymous with "variable." The (functionally similar) "flash" feature in Rails is even more badly named.
If your controller action returns a ViewResult, and you are tempted to put data into TempData, don't. Use ViewData, instead, in this case.
This may lead to a situation where you might need to have a view look into both ViewData and TempData for a key. One example would be if you have an area in your Site.Master to display error messages on the page. It is likely that one controller action will need to send a message to the view via ViewData (because that action returns a ViewResult), while another action will need to send a message to the view via TempData (because that action returns a RedirectResult, as part of a POST/redirect/GET cycle). In this case, you can use code like this in the Site.Master:
<%= ViewData["Message"] ?? TempData["Message"] %>
Under the hood, TempData works by using server-side Session state, so a distributed session cache like Velocity can be used if your application runs on a server farm. Since the lifetime of data put into TempData is limited to the next request, it is fairly easy to use without worrying about growing the session very large, especially since the most common use for TempData is to store a short error string. But the usual rules on what you can put into a Session apply. If you set the Session state mode to out of process, then the data you put in it must be serializable.