To do this, you're going to need a two pronged approach. First, you're going to match the files you want to keep the same. Then, you'll need to have a default for all non-matched pages (ie. the ones you want converted to .shtml). Doesn't sound like a job for RedirectMatch. Probably a job for mod_rewrite though.
Or, you could redirect all your pages to a script file (php, asp, cgi, whatever) that evaluates the request and decides which page to redirect (again) your user to. That would work also.
The example given won't work...
RedirectMatch 301 ^/(.*[^firstpage|secondpage])\.html http://www.example.com/$1.shtml
RedirectMatch uses regular expressions. Anything in the  matches one individual character. So, [abcde] matches one character, either a, or b, or c, or d, or e.
However, [^abcde] matches any character _except_ a, b, c, d, or e.
So, in the example, you'll match any .html page that doesn't have f, i, r, s, t, |, s, e, c, o, d, p, a, g, e as the letter before the period. That is, this regular express should catch abcz.html but not abcf.html.
You can look for a series of pages with the parenthesis operator. Something like: (firstpage|secondpage)\.html would literally match firstpage.html or secondpage.html. However, there's not a mechanism (that I'm aware of) to do a logical NOT on the parenthesis. There's no such thing as: (^firstpage|^secondpage) as the logical NOT operator (the carrot ^) doesn't work outside of brackets  (it's used for something else outside of the brackets).
Hope this helps.