Quantcast
Channel: User Thomas - Stack Overflow
Viewing all articles
Browse latest Browse all 39

Excluding large sets of objects from a query on a table with fast changing order

$
0
0

I have a table of products with a score column, which has a B-Tree Index on it. I have a query which returns products that have not been shown to the user in the current session. I can't simply use simple pagination with LIMIT for it, because the result should be ordered by the score column, which can change between query calls.

My current solution works like this:

SELECT * FROM products p LEFT JOIN product_seen ps   ON (ps.session_id = ? AND p.product_id = ps.product_id )WHERE ps.product_id is nullORDER BY p.score DESCLIMIT 30;

This works fine for the first few pages, but the response time grows linear to the number of products already shown in the session and hits the second mark by the time this number reaches ~300. Is there a way to fasten this up in MySQL? Or should I solve this problem in an entirely other way?


Edit:These are the two tables:

CREATE TABLE `products` ( `product_id` int(15) NOT NULL AUTO_INCREMENT, `shop` varchar(15) NOT NULL, `shop_id` varchar(25) NOT NULL, `shop_category_id` varchar(20) DEFAULT NULL, `shop_subcategory_id` varchar(20) DEFAULT NULL, `shop_designer_id` varchar(20) DEFAULT NULL, `shop_designer_name` varchar(40) NOT NULL, `created_at` timestamp NULL DEFAULT NULL, `product_url` varchar(255) NOT NULL, `name` varchar(255) NOT NULL, `description` mediumtext NOT NULL, `price_cents` int(10) NOT NULL, `list_image_url` varchar(255) NOT NULL, `list_image_height` int(4) NOT NULL, `ending` timestamp NULL DEFAULT NULL, `category_id` int(5) NOT NULL, `last_update` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, `included_at` timestamp NULL DEFAULT NULL, `hearts` int(5) NOT NULL, `score` decimal(10,5) NOT NULL, `rand_field` decimal(16,15) NOT NULL, `last_score_update` timestamp NULL DEFAULT NULL, `active` tinyint(1) NOT NULL DEFAULT '0', PRIMARY KEY (`product_id`), UNIQUE KEY `unique_shop_id` (`shop`,`shop_id`), KEY `score_index` (`active`,`score`), KEY `included_at_index` (`included_at`), KEY `active_category_score` (`active`,`category_id`,`score`), KEY `active_category` (`active`,`category_id`,`product_id`), KEY `active_products` (`active`,`product_id`), KEY `active_rand` (`active`,`rand_field`), KEY `active_category_rand` (`active`,`category_id`,`rand_field`)) ENGINE=InnoDB AUTO_INCREMENT=55985 DEFAULT CHARSET=utf8

CREATE TABLE `product_seen` ( `seenby_id` int(20) NOT NULL AUTO_INCREMENT, `session_id` varchar(25) NOT NULL, `product_id` int(15) NOT NULL, `last_seen` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, `sorting` varchar(10) NOT NULL, `in_category` int(3) DEFAULT NULL, PRIMARY KEY (`seenby_id`), KEY `last_seen_index` (`last_seen`), KEY `session_id` (`session_id`,`seenby_id`), KEY `session_id_2` (`session_id`,`sorting`,`seenby_id`)) ENGINE=InnoDB AUTO_INCREMENT=17431 DEFAULT CHARSET=utf8


Edit 2:
The query above is a simplification, this is the real query with EXPLAIN:
EXPLAIN SELECT     DISTINCT p.product_id AS id,     p.list_image_url AS image,     p.list_image_height AS list_height,     hearts,     active AS available,     (UNIX_TIMESTAMP( ) - ulp.last_action) AS last_lovedFROM `looksandgoods`.`products` pLEFT JOIN `looksandgoods`.`user_likes_products` ulp ON ( p.product_id = ulp.product_id AND ulp.user_id =1 )LEFT JOIN `looksandgoods`.`product_seen` sb ON (sb.session_id = 'y7lWunZKKABgMoDgzjwDjZw1'    AND sb.sorting = 'trend'    AND p.product_id = sb.product_id )WHERE p.active =1AND sb.product_id IS NULLORDER BY p.score DESCLIMIT 30 ;


Explain output, there is still a temp table and filesort, although the keys for the join exist:
+----+-------------+-------+-------+----------------------------------------------------------------------------------------------------+------------------+---------+----------------------------------+------+----------------------------------------------+| id | select_type | table | type  | possible_keys                                                                                      | key              | key_len | ref                              | rows | Extra                                        |+----+-------------+-------+-------+----------------------------------------------------------------------------------------------------+------------------+---------+----------------------------------+------+----------------------------------------------+|  1 | SIMPLE      | p     | range | score_index,active_category_score,active_category,active_products,active_rand,active_category_rand | score_index      | 1       | NULL                             | 2299 | Using where; Using temporary; Using filesort ||  1 | SIMPLE      | ulp   | ref   | love_count_index,user_to_product_index,product_id                                                  | love_count_index | 9       | looksandgoods.p.product_id,const |    1 |                                              ||  1 | SIMPLE      | sb    | ref   | session_id,session_id_2                                                                            | session_id       | 77      | const                            |  711 | Using where; Not exists; Distinct            |+----+-------------+-------+-------+----------------------------------------------------------------------------------------------------+------------------+---------+----------------------------------+------+----------------------------------------------+

Viewing all articles
Browse latest Browse all 39

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>