Running 'testthat.R' [33s]
Running the tests in 'tests/testthat.R' failed.
Complete output:
  > library(testthat)
  > library(bigKRLS)
  Loading required package: bigmemory
  
  
  bigKRLS is authored by Pete Mohanty (Stanford University) and Robert Shaffer (University of Pennsylvania) under License GPL 3.
  
  To get started, check out vignette("bigKRLS_basics").
  
  If you use this package in your research, kindly cite:
  
  
  Pete Mohanty and Robert Shaffer. 2018. 'Messy Data, Robust Inference? Navigating Obstacles to Inference with bigKRLS.' Political Analysis. Cambridge University Press. DOI=10.1017/pan.2018.33. pages 1-18.
  > 
  > # have to disable R_TESTS environment variable or parallel::makePSOCKcluster hangs.
  > Sys.setenv("R_TESTS" = "")
  > 
  > 
  > test_check("bigKRLS")
  ..........................
  All done. You may wish to use summary() for more detail, predict() for out-of-sample forecasts, or shiny.bigKRLS() to interact with results. For an alternative approach, see help(crossvalidate.bigKRLS). Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
  
  
  
  MODEL SUMMARY:
  
  lambda: 0.1306 
  N: 32 
  N Effective: 15.17858 
  R2: 0.9405 
  R2AME**: 0.8477 
  
  Average Marginal Effects:
  
       Estimate Std. Error t value Pr(>|t|)
  cyl   -0.0733     0.0765 -0.9581   0.3806
  disp  -0.0037     0.0022 -1.6485   0.1582
  hp    -0.0145     0.0040 -3.5903   0.0148
  drat   1.2786     0.6303  2.0287   0.0963
  wt    -1.4940     0.3115 -4.7968   0.0045
  qsec   0.3678     0.1741  2.1123   0.0864
  vs*    1.2772     0.6234  2.0488   0.0938
  am*    0.9967     0.5954  1.6739   0.1530
  gear   1.0847     0.2867  3.7828   0.0120
  carb  -0.5951     0.1518 -3.9200   0.0104
  
  
  Percentiles of Marginal Effects:
  
            5%     25%     50%     75%     95%
  cyl  -0.3219 -0.1071 -0.0182  0.0065  0.0406
  disp -0.0108 -0.0080 -0.0040 -0.0011  0.0044
  hp   -0.0372 -0.0258 -0.0138 -0.0009  0.0038
  drat -0.4892  0.6973  1.1559  2.1076  3.4894
  wt   -3.5756 -1.9904 -1.2308 -0.7641 -0.2853
  qsec -0.1250  0.0504  0.2099  0.5900  1.2423
  vs*  -0.3757  0.8754  1.2843  1.8167  2.6408
  am*  -0.6929  0.3432  0.8767  1.8618  2.7406
  gear -0.4504 -0.1866  0.8420  1.9432  3.4149
  carb -1.1902 -1.0159 -0.5844 -0.2336  0.0107
  
  (*) Reported average and percentiles of dy/dx is for discrete change of the dummy variable from min to max (usually 0 to 1)).
  
  
  (**) Pseudo-R^2 computed using only the Average Marginal Effects.
  
  
  You may also wish to use predict() for out-of-sample forecasts or shiny.bigKRLS() to interact with results. Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
  
  Saving model estimates to:
  
   bigKRLS_test_results 
  
  
  0 matrices saved as big matrices (base R save() may be used safely in this case too).
  Smaller, base R elements of the outputted object saved: bigKRLS_test_results/estimates.RData 
  
  Total file size approximately 0 megabytes.
  .
  model estimates will be saved to:
  
   bigKRLS_test_bigmemory_results 
  
  ................
  saving vcovmatyhat to d:/Rcompile/CRANpkg/local/3.6/bigKRLS.Rcheck/tests_x64/testthat
  vcovmatyhat successfully saved to disk (and removed from memory for speed).
  ..........
  saving output to bigKRLS_test_bigmemory_results 
  	writing bigKRLS_test_bigmemory_results/X.txt ...
  	writing bigKRLS_test_bigmemory_results/K.txt ...
  	writing bigKRLS_test_bigmemory_results/vcov.est.c.txt ...
  	writing bigKRLS_test_bigmemory_results/derivatives.txt ...
  
  
   5 matrices saved as big matrices.
  
  to reload, use syntax like:
  
  load.bigKRLS("d:\RCompile\CRANpkg\local\3.6\bigKRLS.Rcheck\tests_x64\testthat\bigKRLS_test_bigmemory_results")
   or
  load.bigKRLS("d:\RCompile\CRANpkg\local\3.6\bigKRLS.Rcheck\tests_x64\testthat\bigKRLS_test_bigmemory_results", newname="my_estimates")
  
  base R elements of the output saved to estimates.RData.
  Total file size approximately 0 megabytes.
  
  
  All done. You may wish to use summary() for more detail, predict() for out-of-sample forecasts, or shiny.bigKRLS() to interact with results. For an alternative approach, see help(crossvalidate.bigKRLS). Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
  
  	Reading from K.txt 
  	Reading from X.txt 
  	Reading from derivatives.txt 
  	Reading from vcov.est.c.txt 
  NOTE: vcov.est.fitted not found in .RData or in big matrix file, vcov.est.fitted.txt .
  
  ............................
  Overview of Model Performance
  
  N: 32 
  Seed: 123 
  
                                                     In Sample Out of Sample
  Mean Squared Error (Full Model)                        1.125         7.532
  Mean Squared Error (Average Marginal Effects Only)   282.697       302.770
  Pseudo-R^2 (Full Model)                                0.971         0.919
  Pseudo-R^2 (Average Marginal Effects Only)             0.860         0.866
                                                                            
  N                                                     26.000         6.000
  
  
  Summary of Training Model:
  
  
  MODEL SUMMARY:
  
  lambda: 0.0749 
  N: 26 
  N Effective: 8.693671 
  R2: 0.9713 
  R2AME**: 0.8595 
  
  Average Marginal Effects:
  
       Estimate Std. Error t value Pr(>|t|)
  cyl   -0.0308     0.0738 -0.4177      NaN
  disp  -0.0013     0.0019 -0.6889      NaN
  hp    -0.0214     0.0036 -5.9445      NaN
  drat   0.7105     0.5552  1.2797      NaN
  wt    -2.1883     0.2698 -8.1106      NaN
  qsec   0.4290     0.1427  3.0069      NaN
  vs*    1.6259     0.5522  2.9445      NaN
  am*    0.8126     0.5254  1.5466      NaN
  gear   1.2481     0.2464  5.0661      NaN
  carb  -0.6583     0.1271 -5.1802      NaN
  
  
  Percentiles of Marginal Effects:
  
            5%     25%     50%     75%     95%
  cyl  -0.2678 -0.0583 -0.0249  0.0639  0.1224
  disp -0.0157 -0.0097 -0.0034  0.0065  0.0148
  hp   -0.0532 -0.0355 -0.0238 -0.0039  0.0019
  drat -0.8121  0.1667  0.6667  1.2357  3.1801
  wt   -4.8466 -2.6870 -1.8917 -1.4949 -0.6539
  qsec -0.1957  0.1062  0.3431  0.7820  1.2412
  vs*   0.1074  1.0071  1.6507  2.2429  3.3543
  am*  -0.8303 -0.0115  0.7399  1.5441  3.1487
  gear -0.4625 -0.0895  1.0600  2.1423  3.4811
  carb -1.4582 -1.2111 -0.5200 -0.2011 -0.0732
  
  (*) Reported average and percentiles of dy/dx is for discrete change of the dummy variable from min to max (usually 0 to 1)).
  
  
  (**) Pseudo-R^2 computed using only the Average Marginal Effects.
  
  
  You may also wish to use predict() for out-of-sample forecasts or shiny.bigKRLS() to interact with results. Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
  
  ........................................
  ......................
  ......................
  ....................
  ................
  .................
  ................
  ................
  ....................
  .....................
  ....................
  ....................
  Overview of Model Performance
  
  N: 250 
  Kfolds: 4 
  Seed: 1234 
  
                          Fold 1 Fold 2 Fold 3 Fold 4
  MSE (In Sample)          0.588  0.759  0.739  0.776
  MSE (Out of Sample)      1.397  0.911  0.873  1.000
  MSE AME (In Sample)      1.842  1.928  2.197  3.013
  MSE AME (Out of Sample)  2.403  1.668  2.145  2.902
  R2 (In Sample)           0.791  0.778  0.799  0.773
  R2 (Out of Sample)       0.729  0.707  0.626  0.679
  R2 AME (In Sample)       0.727  0.730  0.756  0.738
  R2 AME (Out of Sample)   0.759  0.757  0.658  0.729
  
  MSE denotes Mean Squared Error. AME implies calculations done with Average Marginal Effects only.
  
  Summary of Training Model1:
  
  
  MODEL SUMMARY:
  
  lambda: 0.4242 
  N: 187 
  N Effective: 145.6984 
  R2: 0.791 
  R2AME**: 0.7271 
  
  Average Marginal Effects:
  
     Estimate Std. Error t value Pr(>|t|)
  x1   0.8397     0.1971  4.2611        0
  x2   1.8043     0.1999  9.0251        0
  x3   2.2397     0.1963 11.4087        0
  x4   3.0039     0.1925 15.6079        0
  
  
  Percentiles of Marginal Effects:
  
          5%     25%    50%    75%    95%
  x1 -1.9429 -0.3752 0.9473 2.0216 3.7537
  x2 -0.2061  0.9703 1.8933 2.6245 3.8150
  x3  0.0791  1.4120 2.2522 3.0644 4.7562
  x4  0.8860  2.3723 2.9941 3.8705 4.5804
  
  (**) Pseudo-R^2 computed using only the Average Marginal Effects.
  
  
  You may also wish to use predict() for out-of-sample forecasts or shiny.bigKRLS() to interact with results. Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
  
  
  
  Summary of Training Model2:
  
  
  MODEL SUMMARY:
  
  lambda: 0.6276 
  N: 188 
  N Effective: 152.5944 
  R2: 0.7783 
  R2AME**: 0.7298 
  
  Average Marginal Effects:
  
     Estimate Std. Error t value Pr(>|t|)
  x1   0.9054     0.2153  4.2049        0
  x2   1.4723     0.2080  7.0787        0
  x3   2.3005     0.1993 11.5428        0
  x4   3.3161     0.2051 16.1669        0
  
  
  Percentiles of Marginal Effects:
  
          5%    25%    50%    75%    95%
  x1 -1.0968 0.0766 0.9551 1.6229 3.0300
  x2 -1.1023 0.6026 1.5505 2.4868 3.5027
  x3  0.1288 1.5424 2.4145 3.1858 4.3624
  x4  0.3381 2.5274 3.4317 4.2750 5.4594
  
  (**) Pseudo-R^2 computed using only the Average Marginal Effects.
  
  
  You may also wish to use predict() for out-of-sample forecasts or shiny.bigKRLS() to interact with results. Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
  
  
  
  Summary of Training Model3:
  
  
  MODEL SUMMARY:
  
  lambda: 0.6276 
  N: 188 
  N Effective: 152.6962 
  R2: 0.7986 
  R2AME**: 0.7555 
  
  Average Marginal Effects:
  
     Estimate Std. Error t value Pr(>|t|)
  x1   0.9051     0.2122  4.2658        0
  x2   1.4487     0.2067  7.0081        0
  x3   2.2757     0.1915 11.8828        0
  x4   3.0796     0.2000 15.3978        0
  
  
  Percentiles of Marginal Effects:
  
          5%     25%    50%    75%    95%
  x1 -1.4698 -0.1486 1.0052 1.8587 3.1324
  x2 -0.6077  0.6689 1.4685 2.3180 3.3790
  x3  0.2025  1.4459 2.2698 3.1526 4.3790
  x4  0.3165  2.2254 3.2423 4.2299 5.1061
  
  (**) Pseudo-R^2 computed using only the Average Marginal Effects.
  
  
  You may also wish to use predict() for out-of-sample forecasts or shiny.bigKRLS() to interact with results. Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
  
  
  
  Summary of Training Model4:
  
  
  MODEL SUMMARY:
  
  lambda: 0.899 
  N: 187 
  N Effective: 156.6458 
  R2: 0.7728 
  R2AME**: 0.7381 
  
  Average Marginal Effects:
  
     Estimate Std. Error t value Pr(>|t|)
  x1   0.7033     0.1895  3.7114    3e-04
  x2   1.3737     0.1890  7.2700    0e+00
  x3   2.3585     0.1914 12.3216    0e+00
  x4   2.6777     0.1822 14.6986    0e+00
  
  
  Percentiles of Marginal Effects:
  
          5%    25%    50%    75%    95%
  x1 -1.6465 0.0000 0.9521 1.5197 2.4162
  x2 -0.4891 0.6532 1.5508 2.1235 2.9971
  x3 -0.3150 1.4163 2.4109 3.4662 4.6290
  x4  0.2585 1.7607 2.8242 3.7267 4.5394
  
  (**) Pseudo-R^2 computed using only the Average Marginal Effects.
  
  
  You may also wish to use predict() for out-of-sample forecasts or shiny.bigKRLS() to interact with results. Type vignette("bigKRLS_basics") for sample syntax. Use save.bigKRLS() to store results and load.bigKRLS() to re-open them.
  
  -- 1. Failure: Kfolds crossvalidation works (@test_basic_usage.R#148)  ---------
  kcv$folds not equal to kcvbig$folds.
  183/250 mismatches (average diff: 1.73)
  [1]  3 - 4 == -1
  [2]  2 - 3 == -1
  [3]  4 - 2 ==  2
  [4]  2 - 4 == -2
  [5]  3 - 2 ==  1
  [10] 2 - 3 == -1
  [12] 4 - 2 ==  2
  [13] 2 - 4 == -2
  [16] 4 - 2 ==  2
  ...
  
  -- 2. Failure: Kfolds crossvalidation works (@test_basic_usage.R#149)  ---------
  sum(kcv$fold_1$tested$newdata) not equal to sum(kcvbig$fold_1$tested$newdata[]).
  1/1 mismatches
  [1] 123 - 134 == -11.3
  
  -- 3. Failure: Kfolds crossvalidation works (@test_basic_usage.R#150)  ---------
  sum(kcv$fold_2$tested$predicted) not equal to sum(kcvbig$fold_2$tested$predicted[]).
  1/1 mismatches
  [1] 312 - 297 == 14.6
  
  -- 4. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#157
  kcv$R2_is not equal to kcvbig$R2_is.
  4/4 mismatches (average diff: 0.0104)
  [1] 0.791 - 0.774 ==  0.017347
  [2] 0.778 - 0.790 == -0.011866
  [3] 0.799 - 0.787 ==  0.011545
  [4] 0.773 - 0.772 ==  0.000665
  
  -- 5. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#158
  kcv$R2_oos not equal to kcvbig$R2_oos.
  4/4 mismatches (average diff: 0.0416)
  [1] 0.729 - 0.648 ==  0.0816
  [2] 0.707 - 0.690 ==  0.0168
  [3] 0.626 - 0.667 == -0.0411
  [4] 0.679 - 0.652 ==  0.0269
  
  -- 6. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#159
  kcv$R2AME_is not equal to kcvbig$R2AME_is.
  4/4 mismatches (average diff: 0.0126)
  [1] 0.727 - 0.742 == -0.0145
  [2] 0.730 - 0.731 == -0.0010
  [3] 0.756 - 0.732 ==  0.0232
  [4] 0.738 - 0.750 == -0.0116
  
  -- 7. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#160
  kcv$MSE_is not equal to kcvbig$MSE_is.
  4/4 mismatches (average diff: 0.0614)
  [1] 0.588 - 0.734 == -0.1464
  [2] 0.759 - 0.746 ==  0.0130
  [3] 0.739 - 0.695 ==  0.0438
  [4] 0.776 - 0.733 ==  0.0424
  
  -- 8. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#166
  kcv$MSE_oos not equal to kcvbig$MSE_oos.
  4/4 mismatches (average diff: 0.187)
  [1] 1.397 - 1.276 ==  0.1210
  [2] 0.911 - 0.849 ==  0.0617
  [3] 0.873 - 1.195 == -0.3216
  [4] 1.000 - 1.244 == -0.2435
  
  -- 9. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#167
  kcv$MSE_AME_is not equal to kcvbig$MSE_AME_is.
  4/4 mismatches (average diff: 0.371)
  [1] 1.84 - 2.69 == -0.8533
  [2] 1.93 - 2.00 == -0.0674
  [3] 2.20 - 2.54 == -0.3431
  [4] 3.01 - 2.79 ==  0.2221
  
  -- 10. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#16
  kcv$R2AME_oos not equal to kcvbig$R2AME_oos.
  4/4 mismatches (average diff: 0.0461)
  [1] 0.759 - 0.708 ==  0.0505
  [2] 0.757 - 0.769 == -0.0127
  [3] 0.658 - 0.734 == -0.0768
  [4] 0.729 - 0.685 ==  0.0443
  
  -- 11. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#16
  kcv$MSE_AME_oos not equal to kcvbig$MSE_AME_oos.
  4/4 mismatches (average diff: 0.541)
  [1] 2.40 - 3.18 == -0.7736
  [2] 1.67 - 1.61 ==  0.0616
  [3] 2.15 - 3.05 == -0.8998
  [4] 2.90 - 2.47 ==  0.4285
  
  == testthat results  ===========================================================
  OK: 12 SKIPPED: 0 FAILED: 11
  1. Failure: Kfolds crossvalidation works (@test_basic_usage.R#148) 
  2. Failure: Kfolds crossvalidation works (@test_basic_usage.R#149) 
  3. Failure: Kfolds crossvalidation works (@test_basic_usage.R#150) 
  4. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#157) 
  5. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#158) 
  6. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#159) 
  7. Failure: Kfolds test stats, big vs base (batch 1) (@test_basic_usage.R#160) 
  8. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#166) 
  9. Failure: Kfolds test stats, big vs base (batch 2) (@test_basic_usage.R#167) 
  1. ...
  
  Error: testthat unit tests failed
  Execution halted