Disable joblib.memory caching globally during unittest

I use the joblib.Memory module to cache some functions within several modules. The cache is initialized within modules and classes separately.

Module1:

memory = Memory(location='/cache/')

@memory.cache

def heavy_function(...)

    .....

Module2:

memory = Memory(location='/cache/')

@memory.cache

def heavy_function2(...)

    .....

Module3:

import Module1

import Module2



def heavy_function3(...)

    Module1.heavy_function1(...)

    Module1.heavy_function1(...)

    .....

Now I have a unit testing script and I want to disable the usage of the cache globally during the unit testing to make sure everything is correctly computed.
Is this possible without manually disabling it for each module via
Module1.memory.cachedir=None or without deleting the cachedir?

My current solution just patches each memory call manually

unittest1:

from joblib import Memory

import Module1

Module1.memory = Memory(location=None)

...

unittest.run()

unittest3:

from joblib import Memory

import Module1 # need to import module 1 just to disable its memory

import Module2 # need to import module 2 just to disable its memory

import Modul3

Module1.memory = Memory(location=None)

Module2.memory = Memory(location=None)

...

unittest.run()

The more modules I create, the more manual patching up of the Memory I need. I thought there might be a better solution. One work-around is proposed by me below.

edited Nov 22 '18 at 13:30

asked Nov 15 '18 at 11:36

skjerns

18111

why not monkey-patch those Memory instances?

– georgexsh
Nov 22 '18 at 11:20

This is my current solution, but I thought maybe there is another way?

– skjerns
Nov 22 '18 at 13:23

add a comment |

I use the joblib.Memory module to cache some functions within several modules. The cache is initialized within modules and classes separately.

Module1:

memory = Memory(location='/cache/')

@memory.cache

def heavy_function(...)

    .....

Module2:

memory = Memory(location='/cache/')

@memory.cache

def heavy_function2(...)

    .....

Module3:

import Module1

import Module2



def heavy_function3(...)

    Module1.heavy_function1(...)

    Module1.heavy_function1(...)

    .....

My current solution just patches each memory call manually

unittest1:

from joblib import Memory

import Module1

Module1.memory = Memory(location=None)

...

unittest.run()

unittest3:

from joblib import Memory

import Module1 # need to import module 1 just to disable its memory

import Module2 # need to import module 2 just to disable its memory

import Modul3

Module1.memory = Memory(location=None)

Module2.memory = Memory(location=None)

...

unittest.run()

The more modules I create, the more manual patching up of the Memory I need. I thought there might be a better solution. One work-around is proposed by me below.

edited Nov 22 '18 at 13:30

asked Nov 15 '18 at 11:36

skjerns

18111

why not monkey-patch those Memory instances?

– georgexsh
Nov 22 '18 at 11:20

This is my current solution, but I thought maybe there is another way?

– skjerns
Nov 22 '18 at 13:23

add a comment |

I use the joblib.Memory module to cache some functions within several modules. The cache is initialized within modules and classes separately.

Module1:

memory = Memory(location='/cache/')

@memory.cache

def heavy_function(...)

    .....

Module2:

memory = Memory(location='/cache/')

@memory.cache

def heavy_function2(...)

    .....

Module3:

import Module1

import Module2



def heavy_function3(...)

    Module1.heavy_function1(...)

    Module1.heavy_function1(...)

    .....

My current solution just patches each memory call manually

unittest1:

from joblib import Memory

import Module1

Module1.memory = Memory(location=None)

...

unittest.run()

unittest3:

from joblib import Memory

import Module1 # need to import module 1 just to disable its memory

import Module2 # need to import module 2 just to disable its memory

import Modul3

Module1.memory = Memory(location=None)

Module2.memory = Memory(location=None)

...

unittest.run()

The more modules I create, the more manual patching up of the Memory I need. I thought there might be a better solution. One work-around is proposed by me below.

edited Nov 22 '18 at 13:30

asked Nov 15 '18 at 11:36

skjerns

18111

I use the joblib.Memory module to cache some functions within several modules. The cache is initialized within modules and classes separately.

Module1:

memory = Memory(location='/cache/')

@memory.cache

def heavy_function(...)

    .....

Module2:

memory = Memory(location='/cache/')

@memory.cache

def heavy_function2(...)

    .....

Module3:

import Module1

import Module2



def heavy_function3(...)

    Module1.heavy_function1(...)

    Module1.heavy_function1(...)

    .....

My current solution just patches each memory call manually

unittest1:

from joblib import Memory

import Module1

Module1.memory = Memory(location=None)

...

unittest.run()

unittest3:

from joblib import Memory

import Module1 # need to import module 1 just to disable its memory

import Module2 # need to import module 2 just to disable its memory

import Modul3

Module1.memory = Memory(location=None)

Module2.memory = Memory(location=None)

...

unittest.run()

The more modules I create, the more manual patching up of the Memory I need. I thought there might be a better solution. One work-around is proposed by me below.

python unit-testing caching joblib

edited Nov 22 '18 at 13:30

asked Nov 15 '18 at 11:36

skjerns

18111

edited Nov 22 '18 at 13:30

asked Nov 15 '18 at 11:36

skjerns

18111

edited Nov 22 '18 at 13:30

asked Nov 15 '18 at 11:36

skjerns

18111

asked Nov 15 '18 at 11:36

skjerns

18111

asked Nov 15 '18 at 11:36

skjerns

18111

why not monkey-patch those Memory instances?

– georgexsh
Nov 22 '18 at 11:20

This is my current solution, but I thought maybe there is another way?

– skjerns
Nov 22 '18 at 13:23

add a comment |

why not monkey-patch those Memory instances?

– georgexsh
Nov 22 '18 at 11:20

This is my current solution, but I thought maybe there is another way?

– skjerns
Nov 22 '18 at 13:23

why not monkey-patch those Memory instances?

– georgexsh
Nov 22 '18 at 11:20

This is my current solution, but I thought maybe there is another way?

– skjerns
Nov 22 '18 at 13:23

add a comment |

1 Answer
1

active

oldest

votes

One work-around is to set a flag or an environment variable when running tests. Then check for these flags before initializing the Memory:

Module1

import os

memflag = os.environ.get('UNITTESTING', False)

memory = Memory(location= None if memflag else '/cache/')

@memory.cache

def heavy_function(...)

    .....

unittest1

os.environ["UNITTESTING"] = '1'

import Module1

.....

unittest.run()

del os.environ["UNITTESTING"]

edited Nov 22 '18 at 13:51

answered Nov 22 '18 at 13:33

skjerns

18111

add a comment |

Your Answer

StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});

}
});

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53318600%2fdisable-joblib-memory-caching-globally-during-unittest%23new-answer', 'question_page');
}
);

Post as a guest

Name

Required, but never shown

1 Answer
1

active

oldest

votes

1 Answer
1

active

oldest

votes

One work-around is to set a flag or an environment variable when running tests. Then check for these flags before initializing the Memory:

Module1

import os

memflag = os.environ.get('UNITTESTING', False)

memory = Memory(location= None if memflag else '/cache/')

@memory.cache

def heavy_function(...)

    .....

unittest1

os.environ["UNITTESTING"] = '1'

import Module1

.....

unittest.run()

del os.environ["UNITTESTING"]

edited Nov 22 '18 at 13:51

answered Nov 22 '18 at 13:33

skjerns

18111

add a comment |

One work-around is to set a flag or an environment variable when running tests. Then check for these flags before initializing the Memory:

Module1

import os

memflag = os.environ.get('UNITTESTING', False)

memory = Memory(location= None if memflag else '/cache/')

@memory.cache

def heavy_function(...)

    .....

unittest1

os.environ["UNITTESTING"] = '1'

import Module1

.....

unittest.run()

del os.environ["UNITTESTING"]

edited Nov 22 '18 at 13:51

answered Nov 22 '18 at 13:33

skjerns

18111

add a comment |

One work-around is to set a flag or an environment variable when running tests. Then check for these flags before initializing the Memory:

Module1

import os

memflag = os.environ.get('UNITTESTING', False)

memory = Memory(location= None if memflag else '/cache/')

@memory.cache

def heavy_function(...)

    .....

unittest1

os.environ["UNITTESTING"] = '1'

import Module1

.....

unittest.run()

del os.environ["UNITTESTING"]

edited Nov 22 '18 at 13:51

answered Nov 22 '18 at 13:33

skjerns

18111

One work-around is to set a flag or an environment variable when running tests. Then check for these flags before initializing the Memory:

Module1

import os

memflag = os.environ.get('UNITTESTING', False)

memory = Memory(location= None if memflag else '/cache/')

@memory.cache

def heavy_function(...)

    .....

unittest1

os.environ["UNITTESTING"] = '1'

import Module1

.....

unittest.run()

del os.environ["UNITTESTING"]

edited Nov 22 '18 at 13:51

answered Nov 22 '18 at 13:33

skjerns

18111

edited Nov 22 '18 at 13:51

answered Nov 22 '18 at 13:33

skjerns

18111

answered Nov 22 '18 at 13:33

skjerns

18111

answered Nov 22 '18 at 13:33

skjerns

18111

add a comment |

draft saved

draft discarded

Thanks for contributing an answer to Stack Overflow!

Please be sure to answer the question. Provide details and share your research!

But avoid …

Asking for help, clarification, or responding to other answers.

Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Name

Required, but never shown

Name

Required, but never shown

This page is only for reference, If you need detailed information, please check here

搜尋此網誌

Nsryjdtyk