So teilen Sie einen Datensatz in Zug, Validierung und Test basierend auf dem Wert einer anderen Spalte auf ⇐ Python

[Guest]

Ein Datensatz des Formulars angegeben: < /p>

Code: Select all

         date      user   f1     f2       rank   rank_group  counts
0  09/09/2021  USER100  59.0  3599.9    1         1.0       3
1  10/09/2021  USER100  75.29 80790.0   2         1.0       3
2  11/09/2021  USER100  75.29 80790.0   3         1.0       3
1  10/09/2021  USER100  75.29 80790.0   2         2.0       3
2  11/09/2021  USER100  75.29 80790.0   3         2.0       3
3  12/09/2021  USER100  75.29 80790.0   4         2.0       3
2  11/09/2021  USER100  75.29 80790.0   3         3.0       3
3  12/09/2021  USER100  75.29 80790.0   4         3.0       3
4  13/09/2021  USER100  75.29 80790.0   5         3.0       3
3  12/09/2021  USER100  75.29 80790.0   4         4.0       3
4  13/09/2021  USER100  75.29 80790.0   5         4.0       3
5  14/09/2021  USER100  75.29 80790.0   6         4.0       3
4  13/09/2021  USER100  75.29 80790.0   5         5.0       3
5  14/09/2021  USER100  75.29 80790.0   6         5.0       3
6  15/09/2021  USER100  71.24 28809.9   7         5.0       3
5  14/09/2021  USER100  75.29 80790.0   6         6.0       3
6  15/09/2021  USER100  71.24 28809.9   7         6.0       3
7  16/09/2021  USER100  71.31 79209.9   8         6.0       3
6  15/09/2021  USER100  71.24 28809.9   7         7.0       3
7  16/09/2021  USER100  71.31 79209.9   8         7.0       3
8  17/09/2021  USER100  70.43 82809.9   9         7.0       3
7  16/09/2021  USER100  71.31 79209.9   8         8.0       3
8  17/09/2021  USER100  70.43 82809.9   9         8.0       3
9  18/09/2021  USER100  68.65 82809.9   10        8.0       3
< /code>
Angesichts der Tatsache, dass Rank_group angibt, dass der Datensatz 8 Gruppen enthält. Ich möchte mich in einen Drei -Datensatz (Zug, Validierung und Test) mit einer Rate von 70%, 20%bzw. 10%teilen. In diesem Fall würde ich erwarten, dass Train_set alle Zeilen in entsprechenden Rank_group = 1,0,2,0,0,0,0,0,0 enthält. Das Validation_Set enthält alle Zeilen in der entsprechenden RANK_GROUP = 6.0,7.0 und test_set enthält alle Zeilen in entsprechend Rank_group = 8.0.
[list]
[*]train, validation, test = np.split(user_dataset, [int(.7*len(user_dataset)), int(.2*len(user_dataset)), int(.1*len(user_dataset))])

[/list]
Ansatz II: Verwenden von AD-hoc-Split

Code: Select all

        `max_rank_group = user_dataset[rank_group].max()

train_number = round(max_rank_group * train_rate)
validation_number = round((max_rank_group-train_number) * validation_rate)
test_number = round((max_rank_group-validation_number) * test_rate)

print('train_number ', train_number)
print('validation_number ', validation_number)
print('test_number ', test_number)

print(' ')

train_number_frac = train_number % 1
validation_number_frac = validation_number % 1
test_number_frac = train_number % 1

current_train_rank_list = []
if train_number_frac >= 0.5:
current_train_rank_list = range(1, train_number+1)
else:
current_train_rank_list = range(1, train_number)

current_validation_rank_list = []
if validation_number_frac >= 0.5 and (train_number+validation_number+2) < max_rank_group:
current_validation_rank_list = range(train_number, train_number+validation_number+2)
else:
current_validation_rank_list = range(train_number, train_number+validation_number+1)

current_test_rank_list = []
if test_number_frac >= 0.5 and (train_number+validation_number+test_number+2)