Comments on: More import optimisation http://phildawes.net/blog/2004/09/23/more-import-optimisation/ Mostly programming with a few bits of other stuff Tue, 06 Jan 2009 20:57:45 +0000 http://wordpress.org/?v=2.1.1 By: Binary Relations http://phildawes.net/blog/2004/09/23/more-import-optimisation/#comment-8 Binary Relations Fri, 24 Sep 2004 19:21:01 +0000 http://phildawes.net/blog/2004/09/23/more-import-optimisation/#comment-8 <strong>Triple Loading</strong> The one with benchmark results for loading triples into a Redland/MySQL store, including a somewhat pretty diagram. Triple Loading
The one with benchmark results for loading triples into a Redland/MySQL store, including a somewhat pretty diagram.

]]> By: Eric Jain http://phildawes.net/blog/2004/09/23/more-import-optimisation/#comment-9 Eric Jain Sat, 25 Sep 2004 11:22:35 +0000 http://phildawes.net/blog/2004/09/23/more-import-optimisation/#comment-9 If you want to test your system with a really large data set (150M triples), have a look at http://www.isb-sib.ch/~ejain/rdf/data/ :-) I believe the only way to load such amounts of data within reasonable time on reasonable hardware is to make use of the underlying database's bulk loading facilities - I gather you chose a similar approach. We can load 6'000 triples per second, most of which is required for building all the indexes... If you want to test your system with a really large data set (150M triples), have a look at http://www.isb-sib.ch/~ejain/rdf/data/ :-)

I believe the only way to load such amounts of data within reasonable time on reasonable hardware is to make use of the underlying database’s bulk loading facilities - I gather you chose a similar approach. We can load 6′000 triples per second, most of which is required for building all the indexes…

]]> By: Phil Dawes http://phildawes.net/blog/2004/09/23/more-import-optimisation/#comment-10 Phil Dawes Tue, 28 Sep 2004 08:53:39 +0000 http://phildawes.net/blog/2004/09/23/more-import-optimisation/#comment-10 Hi Eric, When you say 6000 triples a second, is this from rdf/xml, or already parsed into some sort of optimized format? Hi Eric,

When you say 6000 triples a second, is this from rdf/xml, or already parsed into some sort of optimized format?

]]>