apriori running out of memory

[khookguy]

I'm running apriori on a dataset with 25,000 rows and 24 columns (features). When I try to run this with min_support set to anything less than 0.6, it runs out of memory. I'm watching the swapfile grow up to about 200G and then Mac OS shuts it down.

itemsets = apriori(df, min_support=min_support, use_colnames=True)

Any suggestions about how to deal with this? Break the problem into smaller parts? Change parameters to allow more swap space, etc?

[rasbt]

I suggest trying fpgrowth instead of apriori. They are both different approaches that lead to the same solution, but fpgrowth is usually more memory efficient. It's a simple drop-in replacement, so you can just swap it in without making any other changes.

[khookguy]

Thanks. Sorry for the delayed response. fpgrowth works for me without memory problems. Where could I read about the differences in the algorithms between apriori and fpgrowth?

[rasbt]

Glad it works! Regarding resources, I don't have any good ones at hand that do a direct comparison, but the wiki page on association rule algorithms has a good overview with references for more details: https://en.wikipedia.org/wiki/Association_rule_learning#Algorithms