How to perform NCT in batches?

[Brooks0303]

Hi Community :)

I am working on a network using stepwise_ggmModSelect with datasets comprising over 60 nodes and several thousand participants. During using bootnet, I encountered memory overflow issues causing code execution failures, likely due to the big model occupancy. With the generous assistance from @SachaEpskamp , I successfully completed stability testing through a chunked approach by dividing the bootstrap into blocks and combining to calculate results.

1. I would like to inquire is batches approach technically okay for NCT?
2. And is the final significance results after merging the chunks still reliable?
3. is it necessary to set the same seed in every loop?

For reference, below is the code inspired by Sacha's advice for bootnet testing:

library("bootnet")
Network <- estimateNetwork(data, default = "ggmModSelect", corMethod = c("spearman"),stepwise = TRUE, nCores = 16)

n_batch <- 10
n_boot <- 100
n_cores <- 16

# Bootstrap n_batch batches:
boot_batches <- list()

for (i in 1:n_batch){
  #is this necessary?
  set.seed(100)
  # Bootstrap:
  boots <- bootnet(Network, nBoots = n_boot, nCores = n_cores)
  # Relabel bootstrap names:
  boots$bootTable$name <- paste("batch",i,boots$bootTable$name)
  # Store to file for backup:
  saveRDS(boots, file = paste0("boot_batch_",i,".rds"))
  # Add to list:
  boot_batches[[i]] <- boots
}


batch_files <- paste0("boot_batch_",1:n_batch,".rds")
boot_batches <- lapply(batch_files, readRDS)

# Take first as base:
boot_combined <- boot_batches[[1]]

# Add the other batches:
for (i in 2:n_batch){
  boot_combined$boots <- c(boot_combined$boots, boot_batches[[i]]$boots)
  boot_combined$bootTable <- rbind(boot_combined$bootTable, boot_batches[[i]]$bootTable)
}

# Now the object can be used as usual:
plot(boot_combined, plot = "interval", order = "sample", split0 = TRUE)

Any suggestions would be greatly appreciated.

Best regards !!!