--- Log opened Thu Apr 12 00:00:37 2012
03:42 < mcallan> if i don't hit any snags, i can release the cross-domain stage tomorrow.  one of the trickiest things i ever coded
03:43 < mcallan> conseo: how are things at your end?
19:40 < conseo> mcallan: cool, i believe this is a very far push in web technologies, right?
19:40 < conseo> i can save state now, now i am wondering how to store the diff messages best
20:09 < conseo> all gathered archive data is static and has to be stored on client as good as possible. these client requests are likely the highest load put on the system. what do you think?
20:27 < mcallan> yes, probably the highest traffic volume
20:29 < mcallan> conseo: i would start by providing just the data needed by the client, which is currently the diff feed.  let the needs of that client determine the initial form of data on the server side.  later it will be fairly easy to change, as necessary
20:29 < conseo> mcallan: true. i was just wondering as all the data is static if it makes sense to store it in the db and not in static json files
20:30 < conseo> but i can do that later. it is just that the data would allow that
20:30 < conseo> it would also allow to only fetch newly parsed posts, so the client can store them and will keep up-to-date
20:33 < conseo> but these posts are not "new" posts, but can be newly added archives as well... so this would involve some storage thoughts
20:35 < mcallan> files are okay if you have a single index, time or whatever.  files have advantage of being easily readable by humans
20:36 < conseo> ok
20:38 < mcallan> or some fixed hierarchy of indeces (forum id, time).  but probably infeasible if you ever want to query on some index that cuts across the hierarchy, e.g. username.  for that you want support of proper db
20:39 < conseo> they also have the advantage of being redistributable in parts and very fast to serve (well, this depends on the db, but static can be parallelized)
20:39 < conseo> yes
20:41 < mcallan> i would store in db.  but i would not define columns for non-indices.  i.e. don't use the db to structure the data, just the indeces.  let the bulk of the data be json or xml or whatever (col data), and just the indeces get separate cols (col time, col forum)
20:42 < mcallan> (if you post a data structure, i can comment)
20:43 < conseo> ok
20:44 < mcallan> look at existing votorola tables.  esp. the user input tables (e.g. votes) have that kind of format
20:45 < mcallan> not sure it's the best example, because there are complexities to it, but: http://zelea.com/project/votorola/_/javadoc/votorola/a/VoterInputTable.html
20:51 < mcallan> come to think of it, this is the simplest table i've coded: http://zelea.com/project/votorola/s/wap/store/StoreTable.java
20:56 < conseo> ok, looking at it
21:00 < conseo> i will have database courses this semester (lectures start on monday)
21:00 < conseo> yet this is not necessary for this task, just mentioning it
21:14 < mcallan> conseo: what version of postgres are you running?  psql --version
21:17 < mcallan> i want to use a db command that only exists in 9.1
21:17 < conseo> psql (PostgreSQL) 9.1.3
21:17 < mcallan> perfect, thx
21:19 < conseo> which command?
21:22 < conseo> (just if it is not to special to explain)
21:22 < conseo> too
21:23 < mcallan> CREATE TABLE IF NOT EXIST http://www.postgresql.org/docs/current/static/sql-createtable.html
21:24 < conseo> ok, sounds handy
23:41 < conseo> i am off for today. gn8
23:44 < mcallan> n8 c
--- Log closed Fri Apr 13 00:00:53 2012