9 replies to this topic

[netnose]

We welcome new web site submissions. Please submit them to:

http://www.netnose.com/add.htm

The reason why we want new submissions is that we need people to submit a site with the search terms that best find the web site. People can determine the appropriate search terms better than automated computers. Volunteer raters will then visit your web site and rate the suggested search terms/phrases.

Any feedback about our beta system (adding and rating) will help make NetNose better. We hope to hear postive and negative comments.

[compar]

Any feedback about our beta system (adding and rating) will help make NetNose better.  We hope to hear postive and negative comments.

I find the limit of 2 word keyword phrases very limiting. I think searchers commonly use 3 word phrases. Also your Title length limitation of 62 characters including spaces is restrictive.

One more suggestion is that you might expand your number of categories and I also think it is possible for a site to belong in two categories. For instance while a site may have stuff for sale it may also be truly informative.

On a more general note how long has the site been up and how many pages do you currently have indexed?

[Alan Perkins]

Search for insurance (copy and paste this URL):

www.netnose.com/cgi-bin/search.cgi?query=insurance&submit=search

What's that second result doing there?

And why don't I get the same results if I do the same search with a referrer - e.g.(click this link)?

[brendan]

Alan,

If you hit refresh, you keep getting different combinations of sites.

It appears as if the order of the ads is randomized somehow. Perhaps netnose can shed some more light on this.

I saw the ad that you mention....it would appear that the ad title is not part of the ranking algorithm.

::brendan::

[netnose]

Compar –

** I find the limit of 2 word keyword phrases very limiting. I think searchers commonly use 3 word phrases **

You can use more than 2 words when searching as I’m sure you know but what I think you are talking about is adding search terms or phrases when you suggest a web site. No worries here – you are given 5 words – so if you want to add a 3 word phrases such as “new york yankees” you would use one slot for “new york” and the second for “yankees – you are then left with 3 more slots for other keywords/phrases like “baseball” , “sports” …etc. I have been hearing talk among our programming staff of “possibly” adding several more slots in the future.

** Your Title length limitation of 62 characters including spaces is restrictive **

This is interesting – I will forward your comment to the programming staff.

** One more suggestion is that you might expand your number of categories and I also think it is possible for a site to belong in two categories. For instance while a site may have stuff for sale it may also be truly informative **

This is also interesting – I have heard this from some other people too – I think there is talk about expanding this…if we keep hearing this from others I think it will happen.

** On a more general note how long has the site been up and how many pages do you currently have indexed? **

Our site has been up for about 2 months – we have over 2 million web sites. We started with a small index on purpose – we want web sites added manually by webmasters or anyone who believes in/likes a web site enough to take the time to add the site and the search terms associated with the site. This is key – people rather than automated computer algorithms should be adding the relevant search terms (to be rated by others). Raters also have the ability to mark a search term/phrase wrong and to suggest a new term/phrase.



Alan –

**Search for insurance... What's that second result doing there? **

The default searching method is “weighted random” – here’s a description from our site:

This display format provides you with a sampling of what the web has to offer. Your results begin with a random sample of the top rated web sites. This is followed by a more complete sampling of web sites that are available for your search.

The NetNose weighted random display format gives you the opportunity to discover new web sites that are usually hidden among thousands of search results in other engines.

Take a look here: http://www.netnose.c...in/settings.cgi?

You can also search 2 other ways plus exclude adult sites, buy sites, etc. We made weighted random our default in order to give all sites a chance to be viewed by search engine users.


Brendan –

**It appears as if the order of the ads is randomized somehow. Perhaps netnose can shed some more light on this **

** I saw the ad that you mention....it would appear that the ad title is >not part of the ranking algorithm **

The ads that you see right now are random – the ads are mainly for NetNose giving people tips on how to rate, why to rate, how to suggest sites, etc. – it’s a new concept and we want people to take a look at the system a little closer. That’s right, the ad titles are not part of the normal ranking algorithm.

Keep the comments coming – feel free to add your own sites or sites that you really like. They will then be put in the rating system for others to review. The more you try things the more feedback you will generate. We appreciate this during our beta.

NetNose

Edited by netnose, 13 August 2003 - 03:19 PM.

[leftbrain]

One nit-picking comment: I very much dislike sites that start their home page title with the phrase "Welcome to...." Remember that the title is used by the browser when bookmarking a site, and anal-retentives like me who make sure everything is in alphabetical order hate to put a site in the W's when its real name begins with an "N." Just my friendly 2 cents worth (and yes, I liked the site enough to bookmark it).

[Alan Perkins]

Netnose, I'm not sure you understood what I was alluding to. Maybe you don't see the same results as me. Let me be a little more explicit.

In a search for insurance in your directory, I've seen the following two sites listed:

www.insuranceprofitsystems.com
www.fast-homeowners-insurance-quote.com

Contrary to appearances those are both adult sites, and they both appear with titles and descriptions of an adult nature within your insurance results.

So, questions:

1) How are titles, descriptions and categories determined for URL submissions to your directory?
2) Why do adult sites get listed when I'm searching for insurance? What kind of algo does that???

[netnose]

leftbrain - good advice - we took it :-)

Alan - yes, I don't see the same results as you because I'm in "weighted random" and the site order keeps changing with each search - now I see what you are trying to tell me. If I were a betting man, I would bet that the original owners of these 2 insurance domains let them expire and the new owners profited off their original traffic. This is a perfect way to show you how our system works. When you see this happen just press the trash can next to the search results. Don't worry, the raters will catch sites like these and either trash them or re-catagorize them.

1) How are titles, descriptions and categories determined for URL submissions to your directory?

http://www.netnose.com/add.htm is now the only way we want to get data into NetNose. We want people to add search terms to web sites rather than depending on automated crawlers just for the reason above. Crawling does not work well when compared to human judgement.

2) Why do adult sites get listed when I'm searching for insurance? What kind of algo does that???

Answered above - be sure to trash irrelevant web sites

Thanks for explaining your finds - keeps us thinking and lets us into the mind of a searcher :-)

Steve

[Alan Perkins]

If I were a betting man, I would bet that the original owners of these 2 insurance domains let them expire and the new owners profited off their original traffic

That doesn't explain how they got into your directory with titles and descriptions that correctly reflect their content, which is adult in nature, nothing to do with insurance. If they had insurance-related titles and descriptions in your directory I could see the sense in what you are saying.

1) How are titles, descriptions and categories determined for URL submissions to your directory?

http://www.netnose.com/add.htm  is now the only way we want to get data into NetNose.  We want people to add search terms to web sites rather than depending on automated crawlers just for the reason above.  Crawling does not work well when compared to human judgement.

OK, but do your editors review listings before adding them to the database? Or can anybody add a listing and it becomes published instantly, without editing?

2) Why do adult sites get listed when I'm searching for insurance? What kind of algo does that???

Answered above - be sure to trash irrelevant web sites

The site titles and descriptions cotained nothing to do with insurance. The only way they could be listed is if the search algo put weight on the keywords in the domain name, without needing to see the keyword in the title or description too. This seems obviously wrong to me, as it leads to the problem I saw. IMO if the word doesn't appear in the title or description, you shouldn't list it.

[netnose]

Alan -

Some of the data in our directory came from our original crawling in order build a small database. So this is data which was crawled from an earlier date. We did a lot of parsing, etc. but this is a limitation of computers vs. humans. We would rather that this data be cleaned up by raters who will trash this site or mark it as adult and mark it as a site with popups - I think I saw these. So the information changed from the original crawling date - when there was probabaly a real insurance site - a quick check of one site on http://www.archive.org shows the old web site - there's an incomplete archive for the other site on archive.org so I don't know what was there exactly.

http://web.archive.o...ance-quote.com/

Again, this is why we don't like crawlers and why we added the trash option and why rating exists - to get rid of sites like this - I've noticed other crawler based engines have this problem - this is what NetNose will fix - problems like these.

On your other question - yes, added sites are reviewed "before" they are added to the database - this will obviously stop spam.

We have considered crawling the web like other engines looking for data - this is an easy task but don't plan on doing this at the present time (except to see if sites are alive, etc.) We feel that to get the proper search terms/phrases directly from humans and then rating those submissions is the best way to get rid of search junk and misleading sites. We don't want to follow the crowd - we want to try and break new ground. As more people get involved in adding and rating we are sure that our "public voice algorithm" will prove itself. Hopefully this answer helps :-)