Posted in General, Semantic Web, Veudas, tagtriples on Nov 16th, 2005 No Comments »
I couldn’t find a way to comment on Benjamins post, so I’ve stuck it here:
What indexing are you using? My tagtriples store schema is basically a table with 4 ids which joins to an (ID,String) table. When it used to be an RDF store this held both literals and URIs.
I found the key to getting […]
Posted in Semantic Web, Veudas, tagtriples on Mar 24th, 2005 3 Comments »
Had an implementation thought on the train this morning:
Instead of implementing a graph as a single id in the 4th column of the quads table, how about implementing it as a range of triple ids. i.e. Each triple is given a sequential id in the 4th column when asserted, and the graphs are […]
Posted in Semantic Web, Python, Veudas on Jan 1st, 2005 4 Comments »
Hit some bad performance hotspots at work yesterday with the new suffix stuff.
The main problem occurs when you do text searches with multiple words, some of which are very popular. The query optimiser frequently makes a bad decision, which boils down to not being able to compute the permutations of the popular word, […]
Posted in Semantic Web, Python, Veudas on Dec 30th, 2004 No Comments »
I’ve made a new release of the Veudas RDF Knowledge manager. This release incorporates the latest veudastore package, along with some UI tweaks and documentation.
Get it from the Sourceforge project page
Posted in Semantic Web, Python, Veudas on Dec 30th, 2004 No Comments »
Text searches were becoming a big bottleneck in the knowledge base deployment at work. The grep solution allows speedup of pure text searches (such as a label search), but doesn’t work so well when the text search is part of a bigger query.
Ideally I would have used mysql’s fulltext indexing capability to index the […]
Posted in Semantic Web, Python, Veudas on Dec 29th, 2004 No Comments »
Made another release of my veudastore triplestore: Fixed a couple of bugs, and added an optimisation - see changelog for details.
Download it (0.2) from the usual place
Posted in Semantic Web, Python, Veudas on Dec 28th, 2004 No Comments »
I keep meaning to perfect these packages, but it doesnt happen so here’s another ‘release-early’ package.
Veudastore is a standalone version of the mysql RDF backend store that runs the veudas knowledge manager. It can be downloaded from the veudas sf project page.
From the readme:
INSTALL:
——–
- Install mysql and create an empty database
USAGE:
——
import veudastore
ctx = veudastore.init()
ctx.setDatabaseConnectParams(USER,PASSWD,DB,HOST)
ctx.createTables()
ctx.importFromURISource(sourceuri, graphuri)
q […]
Posted in Veudas on Oct 29th, 2004 No Comments »
One of the most useful bits of veudas’ functionality is being able to search the knowledgebase for labels (and subproperties) containing a particular string. Unfortunately this is also the slowest operation.
Currently veudas’ ifp store does a table scan over the nodes table for regex searches - very slow (mysql doesnt support a general regex index). […]
Posted in Semantic Web, Veudas on Oct 14th, 2004 3 Comments »
I’ve made some modifications to the table layout used by my experimental smushing store. In particular, I’ve ditched the hashes table, and lumped node hashing in with the literals table. This improves smushing performance as it reduces the number of tables to update (the current biggest overhead when smushing resources on ifp).
Here’s the new layout:
CREATE […]
Posted in Semantic Web, Veudas on Sep 23rd, 2004 3 Comments »
Claire’s out tonight, so another evening spent on bulk rdf importing. Have managed to get the original 120705 statement dataset import down to 77.6 seconds - that’s ~1500 triples a second!
The extra speed was mainly due to removing the need for database URI to id lookups by taking an in-memory copy of the hashes table. […]
Posted in Semantic Web, Veudas on Sep 22nd, 2004 3 Comments »
Spent the evening working on bulk assert speed for my ifp smushing store. Managed to get a 120000 statement import down from 410 seconds to 109 seconds - that’s over 1000 triples per second. Pretty good considering the rdflib rdf/xml parsing takes 59 seconds on its own - I wonder how fast this would be […]
Posted in Semantic Web, Veudas on Sep 11th, 2004 1 Comment »
One of the things I’ve found when dealing with IFP merged data is that you get lots of URIs for the same logical resource.
A good example of this is foaf:Person ‘Dan Brickley’, who is mentioned in lots of peoples foaf files. Because Dan doesnt attribute a URI to himself (and why should he?), quite […]
Posted in Semantic Web, Veudas on Sep 10th, 2004 No Comments »
One of the reasons I need graphs in my ifpstore is that at work I have an application that needs to ensure it is getting its security information from the right source. It uses a store aggregated from multiple sources for its data, and so without graph-matching, any of the other sources could theoretically add […]
Posted in Semantic Web, Veudas on Sep 10th, 2004 No Comments »
Have implemented a basic ‘parser’ using dodgy regexes to split up the components of the query. It supports SOURCE and OPTIONAL, and inlines the constraints with the patterns. Works reasonably well if you stick to the basic format. Hopefully I’ll get round to uploading a demo later today.
Have also added graph management support to […]
Posted in Veudas on Sep 9th, 2004 No Comments »
I think I’m going to bundle my new store as part of my veudas project. It saves me having to start up a new sf project, and currently veudas is pretty tied to the ifpstore’s features (although theoretically you could plug in a different store with the same features).
Posted in Semantic Web, Veudas on Sep 8th, 2004 No Comments »
Currently my smushing-store design doesnt handle unsmushing. This isn’t a big problem for my needs currently, but conversations with Steve Harris indicated that it’s something required quite frequently by his customers.
Given that its not a common thing for me, I need a solution that doesnt affect query speed (e.g. so don’t want to use a […]
Posted in Semantic Web, Veudas on Sep 7th, 2004 1 Comment »
My IFP store design is a copy of the 3store design, but using 32bit ids instead of hashes in the triples table. This enables a single resource to have multiple URIs, which makes IFP smushing much simpler.
ifpstore currently uses 3 tables:
triples table:
- subject 32bit int
- predicate 32bit int
- object […]
Posted in Semantic Web, Veudas on Aug 1st, 2004 No Comments »
FOAF and DOAP both use IFPs instead of URIs to identify resources. Have been considering how to do this in veudas.
Am currently thinking will provide a new version of the resource view which uses IFP prop/object pairs to identify the resource.
Also will look for IFPs in bnodes objects when displaying the resource. If […]
Posted in Semantic Web, Veudas on Jul 29th, 2004 No Comments »
Have been playing with the 3store db format and have written an experimental rdfsql library in python. I did this because I’m not very hot with c and can knock things up much more quickly in python. I’ve implemented optional queries and a remove statement feature (doesnt handle inferred triples), which I needed for […]
Posted in Semantic Web, Veudas on Jul 27th, 2004 No Comments »
Released version 0.5.2 of veudas
