Nsryjdtyk

I am using python-3.x, and I am trying to generate a list of indexes numbers and shuffle them to use them later to select random values from a sample where this sample will have two variables sample size and dimension number, however here how I generate a list of indexes and them I shuffle them:

dimension = 5

sample_size = 100



generate_indexes = itertools.combinations(range(sample_size),dimension)

all_indexes = list(generate_indexes)



# here I do the shuffle

random.shuffle(all_indexes)

the problem when I increased the dimension number it will take a long time to give the result even if the dimension number is 5 it takes very long or it will not proceed.

Is there any way to make it fast?

because I have a multidimensional sample that contains values and I want to select a random number of values form that sample based on the all_indexes...

As pointed out in the comments, you're generating a very large list and then shuffling it. This is not going to be quick but depending on what you actually need, there may be faster ways of getting what you want.

I ran your code on my machine and found that generating the list of all combinations took about 8 seconds and shuffling it took around 75. If you need to increase the dimension, this time will significantly increase, not to mention the memory requirements of storing very large arrays could start to become significant.

If you are not going to need all of the random indexes, you may be better off sampling each time using,

random.sample(range(sample_size), dimension)

This returns a random collection of distinct dimension elements from 0 to sample_size. This took about 0.0001 second to run with your values of dimension and sample_size. If you don't need too many of the random values, then you'll be much quicker (and memory efficient) generating a new one each time.

There are two issues with this that I can see. Firstly you aren't guaranteed that each new sample won't be a repeat of a previous one, but this can be easily resolved by storing them as you go, and checking if they've been used already.

new_sample = random.sample(range(sample_size), dimension)

if new_sample not in random_indexes:

    random_indexes.append(new_sample)

else:

    # Handle this however you need. 

This does add more run time, but again will be faster if you don't need too many of your samples.

The other difference is that the approach you have used generates tuples of elements that are always sorted so (1,2,3,4,5) will be an element of all_indexes, but (5,4,3,2,1) will not. Using random.sample can produce them in any order, so both could occur. If this is an issue, then you will have to resolve this. Perhaps by putting them into a set before adding them to the list:

new_sample = set(random.sample(range(sample_size), dimension))