tag:blogger.com,1999:blog-293958452024-02-28T19:41:08.121+08:00OutlierHan's Ramblings from the outside.Anonymoushttp://www.blogger.com/profile/03716049635372841267noreply@blogger.comBlogger12125tag:blogger.com,1999:blog-29395845.post-22513960777433216592009-01-14T19:11:00.002+08:002009-01-14T19:19:14.147+08:00Wordpad on Windows 7 supports ODFLinkjacking myself, heh: <a href="http://blog.bytecraft.com.my/blog/angch/2009/01/14/wordpad-windows-7-supports-odf">Wordpad on Windows 7 supports ODF</a><br /><br />Got Windows 7 Beta running on VMware, discovered that Wordpad supports ODF, as mentioned by <a href="http://yoonkit.blogspot.com/">yoonkit</a>. Also notice that they're using the <a href="http://en.wikipedia.org/wiki/Ribbon_%28computing%29">Ribbon interface</a> from Microsoft Office 2007.<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://blog.bytecraft.com.my/sites/default/files/wordpad-odf-small.png"><img style="cursor: pointer; width: 400px; height: 300px;" src="http://blog.bytecraft.com.my/sites/default/files/wordpad-odf-small.png" alt="" border="0" /></a><br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://blog.bytecraft.com.my/sites/default/files/wordpad-odf2-small.png"><img style="cursor:pointer; cursor:hand;width: 400px; height: 300px;" src="http://blog.bytecraft.com.my/sites/default/files/wordpad-odf2-small.png" border="0" alt="" /></a>Anonymoushttp://www.blogger.com/profile/03716049635372841267noreply@blogger.com64tag:blogger.com,1999:blog-29395845.post-37675401865123701532008-10-29T19:25:00.003+08:002008-10-29T19:36:54.634+08:00PHP Namespaces<a href="http://phpimpact.wordpress.com/2008/10/28/php-namespaces-controversy/">Succinctly put.</a> I'm a fan of worse is better, but this is pushing it way too much. PHP's like a bunch of infinite monkeys whacking on keyboards, alternatively coming up with brilliant stuff like <a href="http://my.php.net/simplexml">SimpleXML</a>, Wordpress and Drupal, or brillant [<span style="font-style: italic;">sic</span>] stuff like the namespaces fiasco (<a href="http://wiki.php.net/rfc/namespaceseparator">":::" is "hard to parse/easy to be confused"</a>), and magic_quotes and register_globals. Utterly tasteless decisions.Anonymoushttp://www.blogger.com/profile/03716049635372841267noreply@blogger.com3tag:blogger.com,1999:blog-29395845.post-44249331749071594772008-09-10T21:46:00.002+08:002008-09-10T21:54:28.438+08:00FOSS.my in the planning.Overdue for a FOSS event:<br /><br />http://foss.org.my/projects/events/foss.my/FOSS.my<br /><br />Tentative 8 Nov 2008.<br /><br />Will have an IRC meet on it, probably #myoss, freenode, evening 16 Sep 2008. Everyone should join in if possible, even to lurk.<br /><br />See you there.Anonymoushttp://www.blogger.com/profile/03716049635372841267noreply@blogger.com0tag:blogger.com,1999:blog-29395845.post-63689012650084197852008-08-07T00:32:00.002+08:002008-08-07T00:41:42.656+08:00nginx "bug"Encountered some odd drupal+apache2+nginx interaction bug today. Basically, "Transfer-encoding: chunked" was done on the content twice if drupal 404'd. Doesn't seem to trigger outside of drupal, IIRC. Don't know exactly which fault was it, but:<br /><br />http://www.ruby-forum.com/topic/152435<br /><br />described it and nginx author Igor Sysoev gave a patch to fix it. His own development version of nginx doesn't seem to have that patch incoorperated, but the patch made the bug go away.<br /><br />Win. I had hacked nginx .deb for Ubuntu Hardy amd64 at my informal <a href="http://blog.bytecraft.com.my/angch/2008/08/07/nginx-patch">work site</a>.Anonymoushttp://www.blogger.com/profile/03716049635372841267noreply@blogger.com325tag:blogger.com,1999:blog-29395845.post-40827018248377423972008-07-24T14:26:00.006+08:002008-07-24T14:32:22.381+08:00Patch your nameservers!Just patched a number of our nameservers. ( <a href="http://www.doxpara.com/">http://www.doxpara.com/</a> ). Be careful that it's not your <span style="font-style: italic;">content</span> nameservers that matters here, but your own resolver and upstream's nameservers that matters. Check with the tool in www.doxpara.com. Ingenius way to check for vulnerablity, btw.<br /><br />If you don't trust your upstream's dns, run your own patched nameserver, but don't forward queries upstream, but straight to the root servers.<br /><br />P.S. Use opendns or our own patched dns: 202.190.85.116 (temporary while upstream patches their's. I'll remove this in a bit)Anonymoushttp://www.blogger.com/profile/03716049635372841267noreply@blogger.com1tag:blogger.com,1999:blog-29395845.post-48560602515845877192008-06-11T20:45:00.000+08:002008-06-11T20:46:01.656+08:00Caching queries via functional indices in PostgreSQL.Straight to the point:<br /><br /><blockquote><pre><br />foo=# select count(*) from bar;<br />count<br />--------<br />624569<br />(1 row)<br /><br />foo=# explain analyze select count(*) from bar where baz ilike '%some%string%' and quux = '123';<br /> QUERY PLAN <br />------------------------------------------------------------------------------------------------------------------------<br />Aggregate (cost=25846.58..25846.59 rows=1 width=0) (actual time=1543.654..1543.655 rows=1 loops=1)<br /> -> Seq Scan on bar (cost=0.00..25846.53 rows=17 width=0) (actual time=288.667..1543.556 rows=32 loops=1)<br /> Filter: (((baz)::text ~~* '%some%string%'::text) AND (quux = '123'::bpchar))<br />Total runtime: 1543.798 ms<br /></pre></blockquote><br /><br />It takes 1.5 seconds on our sample data to find 32 rows we want out of 624569 rows. If this happens to be a large table, and we often need to run this query, rather than creating some external or internal trigger to cache this query, we can use PostgreSQL's partial indices to do the work for us:<br /><br /><blockquote><pre><br />foo=# create index bar_idx_some_string on bar(id) where baz ilike '%some%string%' and quux = '123';<br />CREATE INDEX<br />foo=# explain analyze select count(*) from bar where baz ilike '%some%string%' and quux = '123';<br /> QUERY PLAN <br />---------------------------------------------------------------------------------------------------------------------------------<br />Aggregate (cost=12.71..12.72 rows=1 width=0) (actual time=0.470..0.473 rows=1 loops=1)<br /> -> Index Scan using bar_idx_some_string on bar (cost=0.00..12.67 rows=17 width=0) (actual time=0.122..0.406 rows=32 loops=1)<br />Total runtime: 0.534 ms<br /></pre></blockquote><br /><br />Now it takes 0.0005 seconds.<br /><br />This also works if in a single transaction or query session, you need to run a number of queries against a table with the similar conditions. Create the index first (give it a temporary name), run the queries, then drop the index.<br /><br />Caveats:<br />Creating the index takes up about same time as the original query, so you need to reuse the conditions to get back your initial investment.<br /><br />Adding or updating to the table takes up a teeny more time. Works best on infrequently updated tables.<br /><br />Kind of useless if the index condition matches a large percentage of the table.<br /><br />You can use some rather complication functions and conditions for your index, but they must be immutable and must not use external info (other tables, time of day).<br /><br />CREATE INDEX locks the table against writes. Use CREATE INDEX CONCURRENTLY on live systems. It comes with it own caveats though.<br /><br />Docs: <a href="http://www.postgresql.org/docs/8.3/static/sql-createindex.html">CREATE INDEX</a>Anonymoushttp://www.blogger.com/profile/03716049635372841267noreply@blogger.com9tag:blogger.com,1999:blog-29395845.post-21569206344103524232007-10-11T20:55:00.000+08:002007-10-11T21:15:57.703+08:00Rant: Stupid OOo hyperlinks.OpenOffice.org 2.3. Type:<br /><blockquote>The quick brown fox http://www.google.com/<br />over the lazy dog.<br /></blockquote>Now click on the end of line after http://www.google.com/. Type "jumps". Now try to remove "jumps" away from the link. You can't.<br /><br />This: http://www.laliluna.de/remove-openoffice-hyperlink.html don't really work properly. It just changes the style so it doesn't <span style="font-style: italic;">look</span> like a link. In HTML speak, it becomes<br /><blockquote><a href="http://www.google.com/">http://www.google.com/</a><a href="http://www.blogger.com/post-edit.g?blogID=29395845&postID=2156920634410352423" style="text-decoration: none;">jumps</a><br />Source:<br /><a href="http://www.google.com/">http://www.google.com/</a><a href="''" style="text-decoration: none">jumps</a><br /></blockquote>instead of<br /><blockquote><a href="http://www.google.com/">http://www.google.com/</a>jumps<br />Source:<br /><a href="http://www.google.com/">http://www.google.com/</a>jumps</blockquote>The original problem is that OOo puts the cursor to the left of the invisible "</a>" when you try to get to the end of the line, not after.<br /><br />Very, very irritating bug that destroys the mental flow.<br /><br />Related problems: <a href="http://www.openoffice.org/issues/show_bug.cgi?id=4364">4364</a> Nothing new. Resolution: "INVALID"? wtf?Anonymoushttp://www.blogger.com/profile/03716049635372841267noreply@blogger.com0tag:blogger.com,1999:blog-29395845.post-87074267821305357782007-08-08T19:03:00.000+08:002007-08-08T20:38:30.089+08:00Back! Random web optimizations.Wow, that's a long time since my last entry. Just gonna dive right in and continue.<br /><br />Anyway, been using using <a href="http://www.getfirebug.com/">Firebug</a> for ages, and now combined with <a href="http://developer.yahoo.com/yslow/">YSlow</a>, I've finally been looking at the performance numbers for some sites and web apps we have. A few interesting tidbits:<br /><ul><li>Apache Bench not that useful at showing real browser experience. Too many pages have external requests that take forever to load. Firebug's "Net" information is more useful in tracking down bottlenecks. YSlow would take that information and tell you where things suck.<br /></li><br /><li>Just bite the CPU load hit and enable transparent deflate compression on Apache. It's worth it. The network bandwidth is a lot more of a bottleneck than CPU processing is.<br /><code>AddOutputFilterByType DEFLATE text/html text/plain text/xml text/css application/x-javascript</code></li><br /><li>Drupal 5.x can automagically concat its many .css together. No need to muck around. Admin - Site Config - Performance - Aggregate and compress CSS files. Too bad it does not handle the nonconformant themes' css.</li></ul>Anonymoushttp://www.blogger.com/profile/03716049635372841267noreply@blogger.com2tag:blogger.com,1999:blog-29395845.post-1154458419353574252006-08-02T01:33:00.000+08:002006-08-02T02:53:39.413+08:00Series generating functions in PostgreSQLI'm not sure how much the following constitutes Functional Programming in SQL, but various aspects of SQL reminds me much of the list and array processing functionalities in perl, python and Unix pipes.<br /><br /><strike><br />Anyway, consider the following clone of python's range() built-in function in PostgreSQL:<br /></strike>Aw, crud. I'm an idiot. PostgreSQL has already a built-in <a href="http://www.postgresql.org/docs/8.1/static/functions-srf.html">generate_series()</a> to cover this. Preserving my original code here though:<br /><pre><br />-- range() to emulate python's range(start, stop, step)<br />-- Note: "immutable" kind of makes this a pure function with no side effects<br />CREATE OR REPLACE FUNCTION range(int, int, int) RETURNS SETOF INT<br />LANGUAGE plpgsql IMMUTABLE AS '<br />DECLARE<br /> i integer;<br /> start alias for $1;<br /> stop alias for $2;<br /> step alias for $3;<br />BEGIN<br /> i := start;<br /><br /> IF step = 0 THEN<br /> EXIT;<br /> END IF;<br /><br /> IF step > 0 THEN<br /> LOOP<br /> IF i >= stop THEN<br /> EXIT;<br /> END IF;<br /> RETURN NEXT i;<br /> i := i + step;<br /> END LOOP;<br /> ELSE<br /> LOOP<br /> IF i < stop THEN<br /> EXIT;<br /> END IF;<br /> RETURN NEXT i;<br /> i := i + step;<br /> END LOOP;<br /> END IF;<br />END;<br />';<br /><br /><br />-- Overloaded range() to emulate python's range(start, stop)<br />-- Written in the "sql" language because pgsql inlines SQL functions (faster).<br />CREATE OR REPLACE FUNCTION range(int, int) RETURNS SETOF INT<br />IMMUTABLE LANGUAGE SQL AS 'select * from range($1, $2, 1);';<br /><br />-- Overloaded range() to emulate python's range(stop)<br />-- Written in the "sql" language because pgsql inlines SQL functions (faster).<br />CREATE OR REPLACE FUNCTION range(int) RETURNS SETOF INT<br />IMMUTABLE LANGUAGE SQL AS 'select * from range(0, $1, 1);';<br /></pre><br /><br />I'm losing my train of thought here, but point is that with generate_series(), and postgresql'S CREATE AGGREGATE as a starting point, you'd be able to construct some pretty functional style SQL programming. <br /><br />One quick, practical use of generate_series() would be to quickly fill in your database with test values:<br /><br /><pre><br />-- Insert 1000 random users, 20-40 years of age,<br />INSERT INTO users (uid, username, date_of_birth)<br /> SELECT <br /> nextval('uid_seq'), <br /> generate_random_username(), <br /> now() - ('1 year'::interval * random() * 20) - '20 years'::interval<br /> FROM<br /> generate_series(1, 1000);<br /></pre><br />which the method we often use at work. More later.Anonymoushttp://www.blogger.com/profile/03716049635372841267noreply@blogger.com45tag:blogger.com,1999:blog-29395845.post-1153155698200320372006-07-18T00:18:00.000+08:002006-07-18T01:01:38.276+08:00Lossy Logic ParadigmData compression algorithms fall into two categories: Lossless and Lossy. The former expects the decompressed data to be exactly the same as the original data while the latter can sacrifice a given amount of precision to achieve greater compression rates, so long as the decompressed data is Good Enough.<br /><br />On the other hand, almost all other algorithms are lossless. For example, <code>bigdatabasetable.getCount()</code> to return <b>exactly</b> the correct, stable, no less of precision count at that time, no matter how many megabytes of information and seconds you need to run through to get that number. Which is annoying, when it returns 1e+6 when the context of the statement is:<br /><br /><pre><br />if (bigdatabasetable.getCount() > 1024) {<br /> // Did we just scanned a terabyte of information? Oops.<br /> cout << "Database is not small!";<br />}</pre><br /><br />If you're using SQL, the above can slightly be "optimized" to<br /> <pre>SELECT COUNT(*) FROM (SELECT 1 FROM bigdatabasetable LIMIT 1024) AS FOO</pre><br />to avoid scanning <i>too</i> much information.<br /><br />What'd be interesting is: <code>bigdatabasetable.getEstimatedCount(100)</code> where 100 would mean "spend up to about 100 msec on this, then give me your best result". In this case, we would be losing precision, but gaining control over the performance of the call.<br /><br />getEstimatedCount is only a example of lossying a given algorithm or function call, but it can be equally applied to many other algorithms. bigarray.getBestMatch(".*?foobar", 100).getFirstTenItems().sort()<br /><br />Applying Lossy Logic would mean we have to build applications to expect a little errors, but we do it all the time in time sensitive applications like video decoders and VOIP, network sensitive apps.<br /><br />This coupled with Asynchronous calls via Message Bus might be an intriguing idea. Too sleepy to detail here now, but this'll allow us to easily make use of spare processors cores, either in the same computer, or nearby...Anonymoushttp://www.blogger.com/profile/03716049635372841267noreply@blogger.com0tag:blogger.com,1999:blog-29395845.post-1152906165259217202006-07-15T02:56:00.000+08:002006-07-18T00:18:40.993+08:00Asynchronous + Message Bus<span style="font-style: italic;font-size:130%;" >Think Asynchronous, not Synchronous; Message Bus, not Point To Point.</span><br /><br />It's a moment of epiphany when I realized the two related concepts are the key to many, many nagging software engineering problems of mine. Neither of them are alien to me, and I'm aware of them for quite a while. It's just that I've never realized how universal the patterns are, and how it applied to so many things. asdfe<br /><br />Consider: how would you elegantly show the progress of a large file copy across several different user interfaces? (e.g. text, GUI, web?) It's tempting to get lost in low level details like <a href="http://www.martinfowler.com/articles/injection.html">Dependency Injection</a>, when it's better to decouple the file copy process and the progress display into two asynchronous thread or process, and then send out periodic progress events to a Message Bus where interested user interfaces can display them. Caveat: this only applies to slow operations. This even allows for more than one user interface (listener) per operation. Traditionally, the file copy operation would be the same process that updates the User Interface's progress bar synchronously.<br /><br />That was just the beginning as I found more and more problems can be elegantly solved by thinking asynchronously. AJAX web interfaces. <a href="http://www.freedesktop.org/wiki/Software/dbus">dbus</a>'s interface between network events and the NetworkManager applet. Qt's <a href="http://doc.trolltech.com/3.3/signalsandslots.html">Signals and Slots</a>. <a href="http://cse.stanford.edu/class/sophomore-college/projects-00/risc/pipelining/index.html">Pipelining</a>. Everyone of them are related.<br /><br />Phrasing their definition to relate to each other:<br /><br /><span style="font-weight: bold;">Asynchronous</span>: issue a request, but don't expect an immediate response. You'll get notified when the response is completed.<br /><br /><span style="font-weight: bold;">Message Bus</span>: don't choose what component to talk to directly, instead send and forget to the bus (aka Message Queue, Channel, Slot), and other interested component<span style="font-weight: bold;">s</span> themselves to choose to listen to the bus.Anonymoushttp://www.blogger.com/profile/03716049635372841267noreply@blogger.com23tag:blogger.com,1999:blog-29395845.post-1152460760670506762006-07-09T23:56:00.000+08:002006-07-18T00:16:54.116+08:00Vernor Vinge MotherlodeFinally found the rest of Vernor Vinge's works at the local book store today (Borders, Berjaya Times Square). Slurged on 4 of them: <span style="font-style: italic;">Tatja Grimm's World</span>, <span style="font-style: italic;">Marooned in Realtime</span>, <span style="font-style: italic;">Collected Stories of Vernor Vinge</span>, <span style="font-style: italic;">The Peace War</span>. Should be fun, Vinge's one of the few writers I dare to buy out right without research.<br /><br />Adding to the pile, I got 3rd Ed. of Stevens' (and Fenner and Rudoff now) <span style="font-style: italic;">Unix Network Programming Volume 1</span>. Had the hardcover 2nd ed, but don't know where that went. This new one is the more expensive (?!) softcover "International Edition".<br /><br />'tis instant noodles for the forseeable future.Anonymoushttp://www.blogger.com/profile/03716049635372841267noreply@blogger.com0