Can we get a general timed LRU built on the workqueue subsys?

Can we get a general timed LRU built on the workqueue subsys?

[David Howells]

Hi Tejun,

Would it be practical to get some sort of timed LRU built on top of the
workqueue such that I can, say, queue an item on it without using the
delayed_work struct?

The reason I'd like this is that I want to avoid using delayed_work because it
would increase the size of the fscache_cookie struct by 50% (110% with
lockdep), and then you'd have to multiply that by the number of cookies on the
system - could be tens or hundreds of thousands; could be a million+ in some
applications.

Really, only one timer should be necessary for the entire queue if every item
in the queue has the same timeout length, since it would only need to keep
track of the item at the front of the queue.  This timer could be managed with
timer_reduce() rather than mod_timer() or del_timer()+add_timer().

Each item in the queue would need a timestamp to say when it expires, say:

	struct work_lru {
		struct work_struct work;
		unsigned long expires_at;
	};

There are other places I could use such a thing too, not just for fscache
cookies.

David

Re: Can we get a general timed LRU built on the workqueue subsys?

[Tejun Heo]

Hello, David.

On Tue, Jun 08, 2021 at 04:14:21PM +0100, David Howells wrote:
> Would it be practical to get some sort of timed LRU built on top of the
> workqueue such that I can, say, queue an item on it without using the
> delayed_work struct?
> 
> The reason I'd like this is that I want to avoid using delayed_work because it
> would increase the size of the fscache_cookie struct by 50% (110% with
> lockdep), and then you'd have to multiply that by the number of cookies on the
> system - could be tens or hundreds of thousands; could be a million+ in some
> applications.
> 
> Really, only one timer should be necessary for the entire queue if every item
> in the queue has the same timeout length, since it would only need to keep
> track of the item at the front of the queue.  This timer could be managed with
> timer_reduce() rather than mod_timer() or del_timer()+add_timer().
> 
> Each item in the queue would need a timestamp to say when it expires, say:
> 
> 	struct work_lru {
> 		struct work_struct work;
> 		unsigned long expires_at;
> 	};
> 
> There are other places I could use such a thing too, not just for fscache
> cookies.

No objection from me but if reducing the size of delayed_work is meaningful
enough I kinda wonder whether this can be generalized so that all
delayed_works are smaller. There's no fundmental reason to have these
smaller ones separate, right?

Thanks.

-- 
tejun