Comments on: Fast String Searching With Suffix Trees http://marknelson.us/1996/08/01/suffix-trees/ Programming, mostly. Thu, 11 Mar 2010 18:54:29 +0000 http://wordpress.org/?v=2.2.3 By: suffix tree resources « KcodeL http://marknelson.us/1996/08/01/suffix-trees/#comment-322899 suffix tree resources « KcodeL Tue, 16 Feb 2010 18:13:02 +0000 http://marknelson.us/1996/08/01/suffix-trees/#comment-322899 [...] http://marknelson.us/1996/08/01/suffix-trees/ [...] […] http://marknelson.us/1996/08/01/suffix-trees/ […]

]]> By: ziman http://marknelson.us/1996/08/01/suffix-trees/#comment-322664 ziman Wed, 10 Feb 2010 11:12:56 +0000 http://marknelson.us/1996/08/01/suffix-trees/#comment-322664 Thanks much for this overview, it definitely helped me much to get an A at today's exam! :) Thanks much for this overview, it definitely helped me much to get an A at today’s exam! :)

]]> By: Its Entirely True » C# Suffix Tree http://marknelson.us/1996/08/01/suffix-trees/#comment-319837 Its Entirely True » C# Suffix Tree Thu, 10 Dec 2009 02:31:12 +0000 http://marknelson.us/1996/08/01/suffix-trees/#comment-319837 [...] I was unable to find a C# implementation of the suffix tree so I ported one I found at Mark Nelson’s Blog. The project of the C# port is located here [...] […] I was unable to find a C# implementation of the suffix tree so I ported one I found at Mark Nelson’s Blog. The project of the C# port is located here […]

]]> By: Legistrate http://marknelson.us/1996/08/01/suffix-trees/#comment-319191 Legistrate Wed, 25 Nov 2009 22:06:06 +0000 http://marknelson.us/1996/08/01/suffix-trees/#comment-319191 Hmm, well it looks like I made an implementation mistake. The algorithm does correctly handle my Phase 11('$'). I am still unsure of how you discovered the proper way to track the position of the active extension, but as the code works, I guess I could read through it a few more times. I do however thing that the suffix pointers added are not always necessary(ie extras), but that could be a mistake too :P. Hmm, well it looks like I made an implementation mistake. The algorithm does correctly handle my Phase 11(’$'). I am still unsure of how you discovered the proper way to track the position of the active extension, but as the code works, I guess I could read through it a few more times. I do however thing that the suffix pointers added are not always necessary(ie extras), but that could be a mistake too :P.

]]> By: Legistrate http://marknelson.us/1996/08/01/suffix-trees/#comment-319181 Legistrate Wed, 25 Nov 2009 19:39:53 +0000 http://marknelson.us/1996/08/01/suffix-trees/#comment-319181 I found your implementation the most useful of the various implementations out there, but the divergence from Gusfield was very confusing at first. Then there are also some things that Gusfield doesn't seem to address directly. For example, If you just wanted to implement suffix links, how would you know what the characters are when you walk up one edge to arrive at the parent (y in the text). You need to know more than just the one character leading from that parent to the edge so you can walk back down after you traverse the suffix link with possibly multiple nodes. I ask because in your code I finally realized that you don't actually walk down the edge to either a newly created inner node or to new children. In this way you have the Suffix start track the position of the char that indicated the edge to the correct node. While I'm still fuzzy on why exactly that works, in general I would say it does. The start being larger than stop signals an external node, and the converse an internal node. So in a phase, that means extensions are applied Rule 1, Rule 2(splits), Rule 2(new kids), Rule 3. But when I use the string "mississippi$" I have an unexpected split in the last phase. In Phase 10 adding (the last)'i' Suffix start is 9. The active point splits ppi$ into p: pi$ and i$, and the active point moves to the root. Thus start is incremented by one(10) and the next extension should be explicit(start==stop) which it is since 'i' is already an internal node off the root. Rule 3 breaks the loop and we move to the next phase('$'). But here, the stop has increased too, and now the algorithm thinks that the active point should be an implicit node. This is not true, so a split is performed on the explicit node, and there is now an extra empty child in the final tree. Could you maybe mention a little of how you devised a way to have a start counter for tracking the relevant index in the string as you move from extension to extension rather than the for loop covering each extension? I found your implementation the most useful of the various implementations out there, but the divergence from Gusfield was very confusing at first. Then there are also some things that Gusfield doesn’t seem to address directly.

For example, If you just wanted to implement suffix links, how would you know what the characters are when you walk up one edge to arrive at the parent (y in the text). You need to know more than just the one character leading from that parent to the edge so you can walk back down after you traverse the suffix link with possibly multiple nodes.

I ask because in your code I finally realized that you don’t actually walk down the edge to either a newly created inner node or to new children. In this way you have the Suffix start track the position of the char that indicated the edge to the correct node. While I’m still fuzzy on why exactly that works, in general I would say it does.

The start being larger than stop signals an external node, and the converse an internal node. So in a phase, that means extensions are applied Rule 1, Rule 2(splits), Rule 2(new kids), Rule 3. But when I use the string “mississippi$” I have an unexpected split in the last phase. In Phase 10 adding (the last)’i’ Suffix start is 9. The active point splits ppi$ into p: pi$ and i$, and the active point moves to the root. Thus start is incremented by one(10) and the next extension should be explicit(start==stop) which it is since ‘i’ is already an internal node off the root. Rule 3 breaks the loop and we move to the next phase(’$'). But here, the stop has increased too, and now the algorithm thinks that the active point should be an implicit node. This is not true, so a split is performed on the explicit node, and there is now an extra empty child in the final tree.

Could you maybe mention a little of how you devised a way to have a start counter for tracking the relevant index in the string as you move from extension to extension rather than the for loop covering each extension?

]]> By: Illya Havsiyevych http://marknelson.us/1996/08/01/suffix-trees/#comment-309141 Illya Havsiyevych Fri, 03 Jul 2009 21:47:56 +0000 http://marknelson.us/1996/08/01/suffix-trees/#comment-309141 Hello, FYI, Some other Suffix Trees based Java Applets: * Generalized Suffix Tree - http://illya-keeplearning.blogspot.com/2009/06/generalized-suffix-trees-java-applet.html * Diff - http://illya-keeplearning.blogspot.com/2009/07/suffix-trees-based-diff-java-applet.html Thanks, illya Hello,

FYI,
Some other Suffix Trees based Java Applets:
* Generalized Suffix Tree - http://illya-keeplearning.blogspot.com/2009/06/generalized-suffix-trees-java-applet.html
* Diff - http://illya-keeplearning.blogspot.com/2009/07/suffix-trees-based-diff-java-applet.html

Thanks,
illya

]]> By: xutao http://marknelson.us/1996/08/01/suffix-trees/#comment-306840 xutao Mon, 15 Jun 2009 21:01:53 +0000 http://marknelson.us/1996/08/01/suffix-trees/#comment-306840 [c] /* input a string to search query [] output start_index and end_index on the string (tree) search against */ void search_tree(char query [], int & start_index, int & end_index) { int start_node = 0; int qp=0; //query position start_index = -1; end_index=-1; bool stop =false; while(!stop){ Edge edge = Edge::Find(start_node, query[qp]); if ( edge.start_node == -1) { stop=true; break; } if (start_node ==0) start_index = edge.first_char_index; print_edge(edge); for (int i = edge.first_char_index; i <=edge.last_char_index; i++){ if(qp >= strlen(query)) { //cout<<"whole query matched"<<endl; stop=true; break; } else if (query[qp] == T[i]){ //cout<<query[qp]<<" "; qp++; end_index = i; } else{ //cout<<"partially matched"<<endl; stop=true; break; } } if (!stop){ //proceed with next node start_node = edge.end_node; if(start_node==-1) stop=true; cout<<"next node "<<start_node; } } } [/c] PLAIN TEXT
C:
  1. /*
  2. input a string to search query []
  3. output start_index and end_index on the string (tree) search against
  4. */
  5. void search_tree(char query [], int  & start_index,
  6.     int  & end_index)
  7.  
  8. {
  9.     int start_node = 0;
  10.     int qp=0; //query position
  11.     start_index = -1;
  12.     end_index=-1;
  13.  
  14.     bool stop =false;
  15.     while(!stop){
  16.         Edge edge = Edge::Find(start_node, query[qp]);
  17.         if ( edge.start_node == -1) {
  18.             stop=true;
  19.  
  20.             break;
  21.         }
  22.         if (start_node ==0) start_index = edge.first_char_index;
  23.         print_edge(edge);
  24.         for (int i = edge.first_char_index; i <=edge.last_char_index; i++){
  25.             if(qp>= strlen(query)) {
  26.                 //cout<<"whole query matched"<<endl;
  27.                 stop=true;
  28.                 break;
  29.             }
  30.             else if (query[qp] == T[i]){
  31.                 //cout<<query[qp]<<" ";
  32.                 qp++;
  33.                 end_index = i;
  34.             }
  35.             else{
  36.                 //cout<<"partially matched"<<endl;
  37.                 stop=true;
  38.                 break;
  39.             }
  40.         }
  41.         if (!stop){ //proceed with next node
  42.             start_node = edge.end_node;
  43.             if(start_node==-1) stop=true;
  44.             cout<<"next node    "<<start_node;
  45.         }   
  46.     }
  47. }

]]> By: xutao http://marknelson.us/1996/08/01/suffix-trees/#comment-306839 xutao Mon, 15 Jun 2009 21:00:41 +0000 http://marknelson.us/1996/08/01/suffix-trees/#comment-306839 The following code is to help query the tree with a string. If the query is a substring of a suffix, it will return the position substring. example code following tree construction [c] int start, end; search_tree(q, start, end); cout= strlen(query)) { cout The following code is to help query the tree with a string. If the query is a substring of a suffix, it will return the position substring.

example code following tree construction
[c]
int start, end;
search_tree(q, start, end);
cout= strlen(query)) {
cout

]]> By: xutao http://marknelson.us/1996/08/01/suffix-trees/#comment-306814 xutao Mon, 15 Jun 2009 18:35:12 +0000 http://marknelson.us/1996/08/01/suffix-trees/#comment-306814 Hi Mark, Good article and useful code. However when I walk_tree for "banana$", it took 2816 iterations! It is certainly nowhere near linear. Please help Hi Mark,
Good article and useful code. However when I walk_tree for "banana$", it took 2816 iterations! It is certainly nowhere near linear. Please help

]]> By: Mark Nelson http://marknelson.us/1996/08/01/suffix-trees/#comment-306426 Mark Nelson Sat, 13 Jun 2009 22:24:09 +0000 http://marknelson.us/1996/08/01/suffix-trees/#comment-306426 @Maria: Check out Ilya's java app and you can see the results of construction in real time. That might be helpful. @Maria:

Check out Ilya's java app and you can see the results of construction in real time. That might be helpful.

]]>