Running examples in ‘bigpca-Ex.R’ failed
The error most likely occurred in:

> base::assign(".ptime", proc.time(), pos = "CheckExEnv")
> ### Name: import.big.data
> ### Title: Load a text file into a big.matrix object
> ### Aliases: import.big.data
> 
> ### ** Examples
> 
> orig.dir <- getwd(); setwd(tempdir()); # move to temporary dir
> # Collate all file names to use in this example #
> all.fn <- c("rownames.txt","colnames.txt","functestdn.txt","funclongcol.txt","functest.txt",
+  paste("rn",1:3,".txt",sep=""),paste("cn",1:3,".txt",sep=""),
+  paste("split",1:3,".txt",sep=""),
+  paste("splitmatCd",1:3,".txt",sep=""),paste("splitmatRd",1:3,".txt",sep=""),
+  paste("splitmatC",1:3,".txt",sep=""), paste("splitmatR",1:3,".txt",sep=""))
> any.already <- file.exists(all.fn)
> if(any(any.already)) {
+  warning("files already exist in the working directory with the same names as some example files") }
> # SETUP a test matrix and reference files #
> test.size <- 4 # try increasing this number for larger matrices
> M <- matrix(runif(10^test.size),ncol=10^(test.size-2)) # normal matrix
> write.table(M,sep="\t",col.names=FALSE,row.names=FALSE,
+  file="functest.txt",quote=FALSE) # no dimnames
> rown <- paste("rs",sample(10:99,nrow(M),replace=TRUE),sample(10000:99999,nrow(M)),sep="")
> coln <- paste("ID",sample(1:9,ncol(M),replace=TRUE),sample(10000:99999,ncol(M)),sep="")
> r.fn <- "rownames.txt"; c.fn <- "colnames.txt"
> Mdn <- M; colnames(Mdn) <- coln; rownames(Mdn) <- rown
> # with dimnames
> write.table(Mdn,sep="\t",col.names=TRUE,row.names=TRUE,file="functestdn.txt",quote=FALSE)
> prv.large(Mdn)
Warning in check.bio() :
  bioconductor does not appear to be installed - this function works better if it is

               colnames 
Row#  rownames ID818179 ID897543  .....  ID349889 
   1 rs1533963   0.2655   0.6547   ...     0.0652 
   2 rs7069442   0.3721   0.3532   ...     0.2774 
   3 rs7629117   0.5729   0.2703   ...     0.6004 
  ..      ....      ...      ...   ...        ... 
 100 rs7286305   0.6049   0.7829   ...     0.6527 
> writeLines(paste(as.vector(M)),con="funclongcol.txt")
> in.fn <- "functest.txt"
> 
> ### IMPORTING SIMPLE 1 FILE MATRIX ##
> writeLines(rown,r.fn); writeLines(coln,c.fn)
> #1. import without specifying row/column names
> ii <- import.big.data(in.fn); prv.big.matrix(ii) # SLOWER without dimnames!
no column names specified
no row names specified
 found 100 column names and 100 marker names

Creating big matrix object to store group data
 predicted disk use:  0 GB
 opening connection to matrix format datafile (1/1): functest.txt

Loading text data into big matrix object:
0         25%         50%         75%         100%
..................................................

 created big.matrix description file: functest.dsc 
 created big.matrix backing file: functest.bck 

Big matrix; 'functest.RData', with: 100 rows, 100 columns
 - data type: numeric 
Warning in check.bio() :
  bioconductor does not appear to be installed - this function works better if it is

              colnames 
Row# rownames     col1     col2  .....    col100 
   1     row1   0.2655   0.6547   ...     0.0652 
   2     row2   0.3721   0.3532   ...     0.2774 
   3     row3   0.5729   0.2703   ...     0.6004 
  ..     ....      ...      ...   ...        ... 
 100   row100   0.6049   0.7829   ...     0.6527 

> #2. import using row/col names from file
> ii <- import.big.data(in.fn,cols.fn="colnames.txt",rows.fn="rownames.txt")
Reading column and row names...
 reading column names from colnames.txt 
 reading row names from /tmp/RtmpKtYlEK/rownames.txt 
 found 100 column names and 100 marker names

 deleting /tmp/RtmpKtYlEK/functest.dsc 

Creating big matrix object to store group data
 predicted disk use:  0 GB
 opening connection to matrix format datafile (1/1): functest.txt

Loading text data into big matrix object:
0         25%         50%         75%         100%
..................................................

 created big.matrix description file: functest.dsc 
 created big.matrix backing file: functest.bck 
> prv.big.matrix(ii)

Big matrix; 'functest.RData', with: 100 rows, 100 columns
 - data type: numeric 
Warning in check.bio() :
  bioconductor does not appear to be installed - this function works better if it is

               colnames 
Row#  rownames ID818179 ID897543  .....  ID349889 
   1 rs1533963   0.2655   0.6547   ...     0.0652 
   2 rs7069442   0.3721   0.3532   ...     0.2774 
   3 rs7629117   0.5729   0.2703   ...     0.6004 
  ..      ....      ...      ...   ...        ... 
 100 rs7286305   0.6049   0.7829   ...     0.6527 

> #3. import by passing colnames/rownames as objects
> ii <- import.big.data(in.fn, col.names=coln,row.names=rown)
 found 100 column names and 100 marker names

 deleting /tmp/RtmpKtYlEK/functest.dsc 

Creating big matrix object to store group data
 predicted disk use:  0 GB
 opening connection to matrix format datafile (1/1): functest.txt

Loading text data into big matrix object:
0         25%         50%         75%         100%
..................................................

 created big.matrix description file: functest.dsc 
 created big.matrix backing file: functest.bck 
> prv.big.matrix(ii)

Big matrix; 'functest.RData', with: 100 rows, 100 columns
 - data type: numeric 
Warning in check.bio() :
  bioconductor does not appear to be installed - this function works better if it is

               colnames 
Row#  rownames ID818179 ID897543  .....  ID349889 
   1 rs1533963   0.2655   0.6547   ...     0.0652 
   2 rs7069442   0.3721   0.3532   ...     0.2774 
   3 rs7629117   0.5729   0.2703   ...     0.6004 
  ..      ....      ...      ...   ...        ... 
 100 rs7286305   0.6049   0.7829   ...     0.6527 

> 
> ### IMPORTING SIMPLE 1 FILE MATRIX WITH DIMNAMES ##
> #1. import without specifying row/column names, but they ARE in the file
> in.fn <- "functestdn.txt"
> ii <- import.big.data(in.fn); prv.big.matrix(ii)
no column names specified
no row names specified
 found 100 column names and 100 marker names

Creating big matrix object to store group data
 predicted disk use:  0 GB
 opening connection to matrix format datafile (1/1): functestdn.txt

Loading text data into big matrix object:
0         25%         50%         75%         100%
..................................................
updated big.matrix rownames from names in file(s)
Warning in import.big.data(in.fn) :
  Had to change description file to a binary file to update rownames. This can be read in with get.big.matrix() [and should be faster to load]

 created big.matrix description file: /tmp/RtmpKtYlEK/functestdn.RData 
 created big.matrix backing file: functestdn.bck 

Big matrix; 'functestdn.RData', with: 100 rows, 100 columns
 - data type: numeric 
Warning in check.bio() :
  bioconductor does not appear to be installed - this function works better if it is

               colnames 
Row#  rownames ID818179 ID897543  .....  ID349889 
   1 rs1533963   0.2655   0.6547   ...     0.0652 
   2 rs7069442   0.3721   0.3532   ...     0.2774 
   3 rs7629117   0.5729   0.2703   ...     0.6004 
  ..      ....      ...      ...   ...        ... 
 100 rs7286305   0.6049   0.7829   ...     0.6527 

> 
> ### IMPORTING SIMPLE 1 FILE MATRIX WITH MISORDERED rownames ##
> rown2 <- rown; rown <- sample(rown);
> # re-run test3 using in.fn with dimnames
> ii <- import.big.data(in.fn, col.names=coln,row.names=rown)
 found 100 column names and 100 marker names

 deleting /tmp/RtmpKtYlEK/functestdn.dsc 

Creating big matrix object to store group data
 predicted disk use:  0 GB
Warning in check.text.matrix.format(fn = ifn, ncol = cls1, header = ID.list[[ffc]],  :
  order of row labels in data file does not match order of inputted rowname list
proceeding, but using name lookup method will make import very slow and possibly unstable
If your source file has an inconsistent order this is the only supported import method.
Otherwise, recommend to cancel (ctrl-C), amend this discrepancy and run again
 opening connection to matrix format datafile (1/1): functestdn.txt

Loading text data into big matrix object:
0         25%         50%         75%         100%
..................................................
Warning in import.big.data(in.fn, col.names = coln, row.names = rown) :
  rownames didn't match what was in filecheck the list in the file at:
 /tmp/RtmpKtYlEK/functestdn_file_rowname_list_check_this.txt

 file preview:
 [1] "rs5629503" "rs2056551" "rs1393927" "rs9031889" "rs8572364" "rs9880503"
 [7] "rs2068755" "rs8749266" "rs3494467" "rs2855364"


 created big.matrix description file: functestdn.dsc 
 created big.matrix backing file: functestdn.bck 
> prv.big.matrix(ii)

Big matrix; 'functestdn.RData', with: 100 rows, 100 columns
 - data type: numeric 
Warning in check.bio() :
  bioconductor does not appear to be installed - this function works better if it is

               colnames 
Row#  rownames ID818179 ID897543  .....  ID349889 
   1 rs5629503   0.0134   0.5112   ...     0.3848 
   2 rs2056551   0.0707   0.5004   ...     0.6367 
   3 rs1393927   0.7774   0.6401   ...     0.5704 
  ..      ....      ...      ...   ...        ... 
 100 rs8949971   0.9447   0.1294   ...     0.0899 

> # restore rownames:
> rown <- rown2
> 
> ### IMPORTING SIMPLE 1 FILE LONG FORMAT by columns ##
> in.fn <- "funclongcol.txt"; #rerun test 2 #
> ii <- import.big.data(in.fn,cols.fn="colnames.txt",rows.fn="rownames.txt")
Reading column and row names...
 reading column names from colnames.txt 
 reading row names from /tmp/RtmpKtYlEK/rownames.txt 
 found 100 column names and 100 marker names

Creating big matrix object to store group data
 predicted disk use:  0 GB
 opening connection to long format datafile (1/1): funclongcol.txt

Loading text data into big matrix object:
0         25%         50%         75%         100%
..................................................

 created big.matrix description file: funclongcol.dsc 
 created big.matrix backing file: funclongcol.bck 
> prv.big.matrix(ii)

Big matrix; 'funclongcol.RData', with: 100 rows, 100 columns
 - data type: numeric 
Warning in check.bio() :
  bioconductor does not appear to be installed - this function works better if it is

               colnames 
Row#  rownames ID818179 ID897543  .....  ID349889 
   1 rs1533963   0.2655   0.6547   ...     0.0652 
   2 rs7069442   0.3721   0.3532   ...     0.2774 
   3 rs7629117   0.5729   0.2703   ...     0.6004 
  ..      ....      ...      ...   ...        ... 
 100 rs7286305   0.6049   0.7829   ...     0.6527 

> 
> ### IMPORTING multifile LONG by cols ##
> # create the dataset and references
> splF <- factor(rep(c(1:3),ncol(M)*c(.1,.5,.4)))
> colnL <- split(coln,splF); MM <- as.data.frame(t(M))
> Ms2 <- split(MM,splF)
> Ms2 <- lapply(Ms2,
+    function(X) { X <- t(X); dim(X) <- c(nrow(M),length(X)/nrow(M)); X } )
> # preview Ms2 - not run # lapply(Ms2,prv.large)
> colfs <- paste("cn",1:length(colnL),".txt",sep="")
> infs <- paste("split",1:length(colnL),".txt",sep="")
> # create multiple column name files and input files
> for(cc in 1:length(colnL)) { writeLines(colnL[[cc]],con=colfs[cc]) }
> for(cc in 1:length(infs)) {
+   writeLines(paste(as.vector((Ms2[[cc]]))),con=infs[cc]) }
> 
> # Now test the import using colnames and rownames lists
> ii <- import.big.data(infs, col.names=colnL,row.names=rown)
reading a single cohort from 3 source files.
 found 100 column names and 100 marker names

Creating big matrix object to store group data
 predicted disk use:  0 GB
 opening connection to long format datafile (1/3): split1.txt

Loading text data into big matrix object:
0         25%         50%         75%         100%
..................................................
 opening connection to long format datafile (2/3): split2.txt

Loading text data into big matrix object:
0         25%         50%         75%         100%
..................................................
 opening connection to long format datafile (3/3): split3.txt

Loading text data into big matrix object:
0         25%         50%         75%         100%
..................................................

 created big.matrix description file: split1.dsc 
 created big.matrix backing file: split1.bck 
> prv.big.matrix(ii)

Big matrix; 'split1.RData', with: 100 rows, 100 columns
 - data type: numeric 
Warning in check.bio() :
  bioconductor does not appear to be installed - this function works better if it is

               colnames 
Row#  rownames ID818179 ID897543  .....  ID349889 
   1 rs1533963   0.2655   0.6547   ...     0.0652 
   2 rs7069442   0.3721   0.3532   ...     0.2774 
   3 rs7629117   0.5729   0.2703   ...     0.6004 
  ..      ....      ...      ...   ...        ... 
 100 rs7286305   0.6049   0.7829   ...     0.6527 

> 
> ### IMPORTING multifile MATRIX by rows ##
> # create the dataset and references
> splF <- factor(rep(c(1,2,3),nrow(M)*c(.1,.5,.4)))
> rownL <- split(rown,splF)
> Ms <- split(M,splF)
> Ms <- lapply(Ms,function(X) { dim(X) <- c(length(X)/ncol(M),ncol(M)); X } )
> # preview Ms - not run # lapply(Ms,prv.large)
> # create multiple row name files and input files
> rowfs <- paste("rn",1:length(rownL),".txt",sep="")
> for(cc in 1:length(rownL)) { writeLines(rownL[[cc]],con=rowfs[cc]) }
> infs <- paste("splitmatR",1:length(colnL),".txt",sep="")
> for(cc in 1:length(infs)) {
+  write.table(Ms[[cc]],sep="\t",col.names=FALSE,row.names=FALSE,file=infs[cc],quote=FALSE) }
> 
> # Now test the import using colnames and rownames files
> ii <- import.big.data(infs, col.names="colnames.txt",rows.fn=rowfs)
no column names specified
 reading row names from /tmp/RtmpKtYlEK/rn1.txt 
 reading row names from /tmp/RtmpKtYlEK/rn2.txt 
 reading row names from /tmp/RtmpKtYlEK/rn3.txt 
reading a single varset from 3 source files.
 found 100 column names and 100 marker names

Creating big matrix object to store group data
 predicted disk use:  0 GB
 opening connection to matrix format datafile (1/3): splitmatR1.txt

Loading text data into big matrix object:
0         25%         50%         75%         100%
......Error in bigVar[nxt.rng[cc], ] <- next.row : 
  object of type 'S4' is not subsettable
Calls: import.big.data
Execution halted