Teh Lack of Speed

From: THERE IS NO GOD BUT (RENDLE) 5 Jul 2006 14:56
To: Peter (BOUGHTONP) 43 of 72
Yes, yes it would. I submit that that doesn't really matter one tiny little bit.
From: Matt 5 Jul 2006 18:49
To: THERE IS NO GOD BUT (RENDLE) 44 of 72

Done all that (the joins I mean). It's not in CVS yet, but it's live here. The messages_get function now only uses POST, USER_PEER (x2) and USER (x2). The join to the thread table now uses the previous call to thread_get() in messages.php and the other joins to the USER table have been moved to separate queries to be done only if the post needs approval or the edit message needs displaying.

 

I've been running a profiler against the code on my own machine (with a copy of the database from here) and noticeable bottle necks are the emoticons code which takes up to 500ms and most noticeably the threads_any_unread() function which is taking at least 1 second to complete(!), compared to the next longest running function being the RSS feed checker which takes 7ms. The threads_any_unread function is usually the cause for the load here. I've tried fiddling with it and adding indexes to the tables and rewriting the query to better use said indexes but it makes diddly squat. It even has a LIMIT clause on it now when it never used to but even that makes no difference, so if it's not the USER_THREAD table that's causing I'm stumped as to what it could be.

EDITED: 5 Jul 2006 18:53 by MATT
From: andy 5 Jul 2006 18:59
To: Matt 45 of 72
The emoticons code as in running that preg_replace against every post at runtime? There any other way to do it? (besides the way it was originally set, converting the emoticons at post-time)
From: Matt 5 Jul 2006 19:28
To: andy 46 of 72
It's the calls to file_exists() in Emoticons->Emoticons() that is the resource hog. It only takes 0.4ms each time to call file_exists() but it's being called quite often. I'm not sure if there is a way around it, maybe initialize the emoticons class within the emoticons.inc.php itself (like the gzipenc.inc.php script does) so when another script includes it it's done automatically. Also maybe don't use file_exists() but rather suppress the error from include() when it doesn't find the definitions.php file.

On my test page, I have 20 posts all with the same number of emoticons and convert() only takes ~35ms to process all 20 of them.
From: andy 5 Jul 2006 19:35
To: Matt 47 of 72
That link just takes me to your portfolio page..
From: Matt 5 Jul 2006 19:45
To: andy 48 of 72
From: Matt 5 Jul 2006 21:47
To: andy 49 of 72
Am I right in thinking this loop below is to weed out any conflicts between the emoticons:

code:
for ($i = 0; $i < $e_keys_size; $i++) {

    for ($j = 0; $j < $e_keys_size; $j++) {

	if ($i != $j) {

	    if (($pos = strpos(strtolower($e_keys[$j]), strtolower($e_keys[$i]))) !== false) {

		$a = $e_keys[$j];
		$b = $e_keys[$i];
		$v = $emoticon[$a];
		$a2 = urlencode($a);

		$a_f = preg_quote(substr($a, 0, $pos), "/");
		$a_m = preg_quote(urlencode(substr($a, $pos, strlen($b))), "/");
		$a_e = preg_quote(substr($a, $pos +strlen($b)), "/");

		$pattern_array[] = "/". $a_f."<span class=[^>]+><span[^>]*>".$a_m."<\/span><\/span>".$a_e ."/";
		$replace_array[] = "<span class=\"e_$v\" title=\"$a2\"><span class=\"e__\">$a2</span></span>";
	    }
	}
    }
}


So it will find :o and :o) and ensure that :o) is matched before :o so :o doesn't replace :o) and leave us with a stray closing bracket and the wrong emoticon showing?

Right?

I'm wondering if the same could be accomplished by simply sorting the emoticon match text array by length with the longest first? Could do that with a single function call see.

Or am I way off base as to what that loop does?
From: andy 5 Jul 2006 22:25
To: Matt 50 of 72
See this is why you're meant to comment your code, Matt. But from staring at it for a bit I'm pretty sure you're right. Which is pretty dumbass, as like you say sorting by length should have the same effect.

And I can access your server through that link. What was I meant to be looking at again? Is that your modified version of the emoticons code or something?
From: Stoo 5 Jul 2006 22:31
To: ALL51 of 72
On a minor note, it's lovely and fast right this very second.
From: Matt 5 Jul 2006 23:13
To: andy 52 of 72

Yes it is the modified code.

 

Emoticons->Emoticons() now executes quicker than the convert function. Hooray.

From: andy 5 Jul 2006 23:47
To: Matt 53 of 72
Ooh it just occured to me why I did that loop. When you preg_replace an array, does text which gets replaced because of earlier entries in the array get considered for text at entries later in the array? It'd make sense that it didn't I guess..
From: Matt 6 Jul 2006 00:29
To: andy 54 of 72
Uh.. I don't know.

We don't use preg_replace anymore though. We use preg_split + PREG_SPLIT_DELIM_CAPTURE now* to find the tags of the emoticons and then the replace is done using in_array and basic PHP code which definitely only performs the replace once so would that matter or not?

* preg_split limit of 100 only matches 100 emoticons maximum vs. preg_replace limit of 100 matches 100 of each array value, which is bad.
From: andy 6 Jul 2006 01:00
To: Matt 55 of 72
If you're ordering them so the longest emoticons are first then it sounds like it shouldn't matter, good work!
From: Ben (BENLUMLEY) 6 Jul 2006 01:10
To: Matt 56 of 72
care to share the name of said profiler?
From: THERE IS NO GOD BUT (RENDLE) 6 Jul 2006 09:17
To: Matt 57 of 72
I'll take a look at the threads_any_unread function...
From: Matt 6 Jul 2006 10:42
To: Ben (BENLUMLEY) 58 of 72
XDebug and WinCacheGrind to examine the profiles it creates.
From: Dave!! 7 Jul 2006 11:18
To: ALL59 of 72

Just had an hour if it being virtually dead including an error from the server which was:

 

Fatal error: Maximum execution time of 30 seconds exceeded in /var/www/vhosts/tehforum.co.uk/httpdocs/forum/include/emoticons.inc.php on line 103
Call Stack
# Function Location
1 {main}() /var/www/vhosts/tehforum.co.uk/httpdocs/forum/messages.php:0
2 message_display() /var/www/vhosts/tehforum.co.uk/httpdocs/forum/messages.php:386
3 message_split_fiddle() /var/www/vhosts/tehforum.co.uk/httpdocs/forum/include/messages.inc.php:618
4 emoticons->emoticons() /var/www/vhosts/tehforum.co.uk/httpdocs/forum/include/messages.inc.php:477
5 count () /var/www/vhosts/tehforum.co.uk/httpdocs/forum/include/emoticons.inc.php:103

 

Not sure if it's any use, but I thought I'd post it just in case :)

From: Manthorp10 Jul 2006 22:37
To: Matt 60 of 72
Don't know if you've been tinkering, Matt, but it's been significantly speedier today and tonight.
From: Drew (X3N0PH0N)10 Jul 2006 22:45
To: Matt 61 of 72
Same.
From: Matt10 Jul 2006 23:30
To: Manthorp 62 of 72
Some of the tweaks I've made to BH are in use here now and I've got some more to come as well, but I think it's mostly down to finding a balence between Apache and MySQL.