New sites often receive a "bump" in search rankings at first, then they float to their natural level in the search results. I suppose it's possible you were seeing that search traffic as a result of the "new site bump" and now your pages are appearing at more the natural level for a brand-new site.
Can you think of something else that might have been going on outside of your site (some breaking news story that might have affected search traffic for keywords important to your site, maybe)?
The thing is, search traffic does fluctuate all the time. You're noticing it a lot right now because you likely don't have as much traffic from other sources, so even a small variance in search traffic shows up huge. (For example, on my employer's website a drop of 50 visitors wouldn't even be noticed most days.) But these kinds of fluctuations are actually par for the course for many sites, and even more so for new sites.
If you're confident that what you're doing is best for the visitor -- that your content is high quality, that the site is laid out for user-friendly navigation, that your inbound links are from credible sources, etc. -- then keep on with what you're doing. Work to build links from other sites that can send you direct, targeted traffic. Build your brand. Keep in mind, search traffic is at best a "loan" that can be recalled any time, for any reason, so focus your attention on getting good steady traffic from other sources.
Good luck with your new site!