EuroForMix

Archived versions are found below.

For newer versions, see changes in https://github.com/oyvble/euroformix/blob/master/NEWS.md

Updates in euroformix_3.0.3 (06.05.20)

Important bug fix in the fitgammamodel function (bug introduced in v3.0.0):
• The returned degradation parameter value was not correct (always overestimated). This influences the degradation slope plot in the EFM GUI. Also, the fix may cause faster convergence of MLE.

Minor issue fixed :
• GUI crashed when loading settings after saving settings with old projects (created with a version before v3.0.0). Thanks to Peter Gill for pointing out this issue.

Updates in euroformix_3.0.2 (06.05.20)

Minor issue fixed :
• The MCMC simulations failed if first random startpoint gave a likelihood of zero (introduced in v3.0.0). Thanks to Eugenio Alladio and Maria Martin Agudo for reporting this issue.

Updates in euroformix_3.0.1 (02.05.20)

Minor bug fix :
• Markers containing space caused problem when loading Marker settings after saving (reading from the configMarkerSetup file failed).

Updates in euroformix_3.0.0 (08.04.20)

See EuroForMix_v3.0.0_changes.txt file below.

Updates in euroformix_2.3.1 (12.02.20)

• Regretting v2.3.0 change in function Qassignate: The Q-allele is not always added when doing Q-assignation anymore ("if-case" at line 62 included again). The previous change caused problems for markers with very few alleles in population.
• Verifiler Plus has been included as a kit.

Updates in euroformix_2.3.0 (05.12.19)

Important changes:
• Fixed bug (L101 in file prepareC.R) when analysing MPS-LUS+ data with stutter model (thanks to Peter Gill).
• References treated as related (under relatedness module) are no longer counted as typed twice when considered in Hp but not Hd. This caused too conservative LRs.
  - The formulaes in function calcGjoint has been modified to not overcount. Also, the prepareC function ensures that the related reference is either a contributor or a known non-contributor.

Minor changes:
• The option "All knowns" in relatedness options was removed due to possible confusion.
• Points in validMLEmodel plots were not shown when having more than 25 markers. Identical point types for different markers will now occure when this happens.
• The user can now also see file all files (not only Rdata extension) when loading R-project (optional).
• Fixed bug in plotTopMPS when missing alleles when analysing with kit-info (eg. LUS/LUS+ format using ForenSeq kit).
• plotEPG2/plotMPS2/plotTopEPG2/plotTopMPS2 now returns the created plotly widget (for later use).
• validMLEmodel function updated:
  - Extra arguments of whether plotting/print should be done.
  - Function now returns cumulative probabilities and p-values from test for each alleles.
• The order of the genotype names used in/returned from the calcGjoint function has now changed for consistency.
• Updated genDatasets function to check that frequency elements are simplex (otherwise an error is thrown).
• In function Qassignate: The Q-allele is now always added when doing Q-assignation ("if-case" at line 62 removed).

Updates in euroformix_2.2.1 (03.09.19)

Minor changes:
• Fixed bug in plotEPG2/plotTopEPG2 when no corresponding decimal of alleles was found in the kit-outcome (getKit).
• New created project files becomes much smaller in size.
• "Set number of random startpoints" in Optimization has changed name to "Set number of required optimizations".

Updates in euroformix_2.2.0 (22.07.19)

Added functionalities:
• The function plotTopLUS has been added. Designed to show the expected coverage reads for MPS based data in LUS format.
• The functions plotEPG2 and plotMPS2 have been added, both using the plotly library to get interactive plots using the browser.
• The functions plotTopEPG2 and plotTopMPS2 have been added, both using the plotly library to get interactive plots using the browser. Designed to show the expected coverage reads for MPS based data in (grouped) sequence format.

Minor changes:
• The degradation option is only deactivated if kit is not selected.
• The stutter option is only deactivated if all alleles are strings.
• Kit selection = "NONE" is now possible (in addition to "empty").
• A warning message is given when the selected kit does not contain loci as provided in evidence.
• When estimating the drop-in model: The estimation now filters out all PHs below selected detection threshold.
• The detection threshold is now also shown in View Evidence plots.

Bug fixes:
• Fixed the bug causing project files to be large after many analyses:

- The eval function did not refer to the current environment "mmTK" when building the stutter prior density function "pXi".

• Fixed the bug causing error when "view references" of partial reference profiles.

Updates in euroformix_2.1.0 (26.02.19)

Changes:
• A bug was fixed in the function prepareC at line 137 (earlier present in v2.0.2-v2.0.4):
  - This has only a potential small effect when applying the stutter model. Alleles given with "." did not earlier find corresponding backward-stutter.
• The "NGM Detect" kit has been added.
• The maximum likelihood method of qualitative LR has been made more robust:
  - Optimizer now uses start values of 0.1,0.35,0.7 (instead of only 0.1).

Minor changes:
• Removing peak heights in reference object if these have been included in the reference file (see line 927 in efm.R).
• The "continuous" terms are replaced with "quantiative".
• Parameter names changed in "Generate data".
• The maximum log-likelihood value for the qualitative method is now printed in the R-console.

Updates in euroformix_2.0.4 (06.11.18)

Minor changes:
• In function prepareC: Bug fixed when finding fragment length for non-numerical alleles (e.g. amel variants).
• In function plotEPG: Bug fixed when plotting a evidence profile with a full dyer dropout.
• In function plotTopEPG: Bug fixed when plotting a evidence profile witha a full dyer dropout.

Updates in euroformix_2.0.3 (12.09.18)

Minor changes in function efm:
• Bug fixed when project are loaded and the button "Model fitted P.H." is clicked.
• When saving projects: The compression level is set to 2 to reduce read/write time. The file sizes will be about the same.

Updates in euroformix_2.0.2 (04.09.18)
See also the file Description about EuroForMix version 2 update below.

Major changes:
• The possibility to include a known relationship under Hd has been added. Here the 1st of the unknown contributors under Hd is given with a specified relationship to a selected reference profile.
• The relationship update effects all analysis done with the quanttitative model (LR,deconvolution,non-contributors simulations etc), except of the Generate Data function. The qualitative LR module has not been updated and does still not support relationship (users can instead use LRmix Studio for this).
• In function efm:

- Allele frequencies has been added to the report.
- The user can now specify a minimum allele frequency for new alleles not found in the frequency database.
- Allele frequencies are no longer normalized when new alleles are included. This will have an impact on the LR if observed alleles are missing in the population frequency data.

Added function:
• calcGjoint(popFreq,nU,fst,refK,refR,ibd) Calculates the joint probability of the genotype of nU unknowns where 1st unknown is given with a specfied relationship (given by ibd) to a reference with alleles refR. Assumes a specific fst value with refK previously typed alleles (known non-contributors).

Function updates:
• contLikMLE: Added arguments: knownRel,ibd.
• prepareC: Added arguments: knownRel,ibd,incS. allAbpind is renamed as allASind to reflect that the vector conserns Allele-Stuttering information. Effects functions where .C is called (contLikMLE,contLikINT,contLiki, deconvolution)
• Qassignate: Added arguments: minFreq,normalize.
• contLikINT: Added arguments: knownRel,ibd.
• getGlist: Added arguments: refR,ibd,refK,fst. Now calls the function calcGjoint to include kinship/fst in genotype probability.
• loglikgammaC (in C-code): Instead of using pG (prob genotypes) for fst=0, pG is now given as the genotype prob of the 1st unknown contributor. This is used to take into account relationship where the 1st unknown is given as the related individual.

Non-contributor/Tippet analysis:
• Random non-contributors are drawn with genotype probabilities for the 1st unknown under Hd (relationship/theta-correction included).

Deconvolution:
• The joint probability of G does now follow the specified model (given theta/relationship), instead of only a HW assumption.

Changed settings:
• Default number of tippets set to 100.
• Default number of MCMC simulations set to 1000.

Minor changes:
• In function efm: Setting values in all settings is now robust: The user gets a message if the input is wrong format Prev. versions crashed if e.g. "0,01" were given instead of "0.01".
• In function freqImport: Importing allele freq files with only 1 marker is now possible. Argument drop=FALSE added to subset of matrix.
• In function efm: Better Compression of files: compress="xv" is used. This requires the save/load to use more time in tradeoff of smaller files.
• In function prepareC: Bug fixed:
  - Added line to avoid being LUS if system dropout
  - isLUS <- isLUS & any( sapply(samples,function(x) sapply(x,function(y) length(y$adata)))>0 ) #ADDED: don't consider LUS format if system dropout

Updates in euroformix_1.11.4: (18.06.18)

• EFM is now compatible with the 32-bit architect of R.
• Bug fixed in efm at line 1136: There was a bug causing the program to crash when showing the EGPs when any of the evidence sample(s) contained a locus dropout.

Updates in euroformix_1.11.3: (04.06.18)

• EFM can now be run in R-version 3.5.
• EFM now follows same version as the R-package (updated to v1.11.3).
• EFM now contains a "fit drop-in model" module. The user can now import a datafile including drop-in peak height and EFM will automatically fit the lambda parameter and show the fitted model against the observations in a histogram.
• EFM handles the LUS nomeclature for MPS using the ForenSeq kit. Specialized plots are provided showing alleles on RU_LUS format.
• Alleles given with separator "_" is handled, a general hierarchy on the LUS, "a_b_c_d..", is possible. Here a-1_b-1_c_d is potential stutter, whereas the RU "a" is extracted for base pair info (degradation).
• Updated for the functions efm, Qassignate, prepareC: Checks if LUS-format requiring all alleles given with separator symbol "_".
• The R-package "snow" is not longer required, the package parallel is used instead.
  - In functions contLikMLEpara and contLikMCMCpara the type of paralellization was changed to PSOCK. Reason was that R-version 3.5.0 did not work with SOCK.
• Update in function Qassignate:
  - When stutters=ON, function checks allele names of type "x_y...". Stutter conducted as -1_-1.
• Update in function prepareC:
  - When stutters=ON, function checks allele names of type "x_y...". Stutter conducted as -1_-1.
  - When degradation=ON, function checks allele names of type "x_y...". Finding base pair of "x" in kitinfo.
• Update in function efm:
• Checks that all allele names are given as "x_y...". If all alleles are on this format, then a LUS based "EPG" like plot is created.
• Stutters/Degradation deactivated for non-STRs (when an empty kit is given).
• The popFreqs from the default folder is automatically imported when right-clicking the "Import from folder" button.
• The random match probability (RMP) and 1/RMP for each references are shown if frequencies are provided. The value of fst is taken into account based on the Balding-Nichols formula. RMP=0 if any alleles are missing in the frequencies.
• A button for evaluating drop-in data automatically has been added to "import data" panel.
• The ML based LR of LRmix variant has been implemented: Added button "Calculate LR" under "MLE based" on the Qual.LR panel.
• The project saving now compresses the Rdata file. This increases the time taken to save/load projects.
• Update in function simDOdistr:
  - Alleles can now also be strings.
• Bugs:
  - In function prepareC at line 109 and 116: The "tmp" variable must be added instead of substituted. The old version did not succeed in counting alleles in references (if more than one reference). Thanks to Kevin Cheng, John Buckleton and Dr Jo-Anne Bright working with the analysis leading to this fix.
  - In function efm at line 2341-2342?? object "optint" should be "opt".
  - In freqImport function: Numerical zeros ("0") in allele freq tables taken account for: EFM interpret these as not observed (NA) instead of having apriori frequency as 0.
  - In efm when allele names are non-numbers (as for MPS data), the Qual based LR crashed. This is now fixed.
• Minor changes:
  - The value under “Set max number of iterations" is now set to 100 instead of 50 by default. Notice: 50 iterations may be too small to obtain achieve the optimal point.
  - The number of contributors can now be changed in a drop-down panel (here maximum is 4, but numbers can be changed).
  - In PP-plot: "1-Envelope-coverage" instead of "Envelope-coverage"
  - Function Qassignate: Now also returns samples given as argument.
  - Function tableReader: header boolean has been added as argument to function.
  - Function getGlist: Warning created by as.numeric is supressed
• Added function to the euroformix R-package:
  - fitgammamodel: A function for fitting a gamma distribution based on the sum peak heights and returns optimized model parameters.

Updates in euroformix_1.10.0 (06.02.18)

• Updated functions: efm,contLikMLE,contLikMLEpara,validMLEmodel
• Functions added to euroformix R-package: plotTopEPG, freqImport, tableReader
  - The helpfunction tableReader within efm.R was moved and created as a seperate function, see ?tableReader.
  - freqImport requires the R-packages XML and RCurl.
• Important notes:
  - Don't use "Select STR kit" if it is not defined in the list (i.e. non-STRs).
• In the GUI (efm):
  • efm changed how frequencies are imported (using the function freqImport).
  • The kit can now be selected regardless of the population frequencies (only overlapping markers will be considered in the model).
  • One of three options (each given as GUI buttons):
    1) Can be given as a single file. The file name will give the population name.
    2) Can be given as a folder with files. The file names will give the population names.
    3) Can be imported directly from the STRidER webpage. The content from the webpage decides the population names.
  • The number of contributors in "Model specification" panel can now also be selected in either a drop-down panel.
  • A restart button "RESTART" has been added to the "Import data" panel. The user can now restart EuroForMix as a clean session.
  • The button-name "All results" is change to "Create report".
  • The R-version, Selected Kit and Selected Population have been added to the report.
  • The button "Model fitted P.H." has been added under "Further Action" in MLE fit panel.
    - This shows the expected peak heights of the top deconvolved genotypes superimposed with the observed peak heights (the function supports multiple evidences).
  • A signif.-level is required to be set from the user when running "Model validation" in "MLE fit".
    - This denotes the significance level used for the envelope-coverage in "Model validation". Default value is 0.01 .
  • Added "Set max number of iterations" in "Optimization"
    - The user can select maximum number of iterations for the optimization. This can reduce the timeusage drastically when the optimization method converges slowly. This changes the variable "iterlim" in nlm. Default is 50.
  • Added "Set likelihood-scaling to avoid zero" in "Integration"
    - The user can select a scaling value of likelihood function used with integration NOTE: scaleINT necessary to tweak to avoid zero-likelihood values for integration. Default value is 700.
• The function plotTopEGP has been added.
  - This function shows the expected peak heights of the top deconvoluted genotypes. The peak heights for corresponding alleles are superimposed.
  - This function requires an MLE-fitted object.
• The function freqImport has been added:
  - This function makes importing frequency files easy: Different formats are allowed.
• In function contLikMLE and contLikMLEpara:
  - The variable "maxIter" has been added an input parameter to the functions contLikMLE and contLikMLEpara:
  - The maximum number of iterations for the optimization nlm (variable of nlm::iterlim=maxIter)
• In function validMLEmodel:
  - The variable "alpha" has been added as an input parameter to the functions validMLEmodel (the significance level used for the envelope-coverage).
• Minor changes:
  - The helptexts for the "Import frequency" buttons have changed.

Updates in euroformix_1.9.3: (09.09.17)

• The function contLikMCMCpara has been added to the R-package:
  - It performs parallelization of the contLikMCMC function.
• For LRsensitivity calculation:
  - The Hp/Hd plots for LR sensitivity has been removed.
  - Parallell processing is now performed by contLikMCMCpara to carry out more samples, using same number of parallels as the number of cores.
• Added flexibility:
  - Allele names (not being AMEL) can now be strings. For these situations the stutter and degradation models will not work.
  - If kitname is not recognised, intensities for the samples are still plotted (sum per loci). In addition a goodness of fit QQ plot is given to check whether the intensities can be assumed being independent gamma-distributed.
• For SNPs:
  - Degradation/stutters turned off automatically.
  - The allelenames can be strings.
• In the loglikgamma.cpp code:
  - The value "INFVAL = std::numeric_limits::infinity()" is used instead for "log(0)".
  - The error caused by the function tgamma_lower(x,y) is now handled by adding a "try" block. Small/High non-negative values of x and y caused unexpected error.

Updates in euroformix_1.9.1: (15.06.17)

• EFM changed from v1.8 to v1.9:
  - All markers in evidence are utilized even when references have missing markers. Implemented for Weight-of-Evidence, Deconvolution and Database searching.
  -> This indicates that users of EuroForMix should describe how they treat missing loci for references (i.e. whether they deselect evidence markers OR include missing markers for the references).
  - The version of euroformix R-package now follow the GUI version. I.e. the R-package starts from 1.9.0 for this current version (v1.9).
• New feature for Generating data: degradation is now possible by adjusting the beta parameter (degradation slope).
  - beta has been added to Parameters in "Generate data"
• New features for Deconvolution:
  -> Genotypes of references for missing markers are also inferred.
  -> The "Joint" table has been replaced.
  -> Marginal probability of each alleles also calculated.
• New check feature for allele frequencies:
  - When EuroForMix starts, all allele frequency numbers for the selected allele frequency file are investigated. A warning message is given if any of these scenarios occur:
  -> Being negative.
  -> If a marker did not provide any allele frequency.
  -> If the sum of the allele frequncies are different from one (up to two decimals).
• Cosmetic changes:
  - In Model validation: The main title now shows "PP-plot under Hp" under Hp and "PP-plot under Hd" for Hd.
  - "Plot EPG" in model setting changed to "Show"
  - "Model options" has been removed in Model specification for "Generate data"
• Changes in functions:
  - Updated validMLEmodel: Also prints allele names for outliers.
  - Updated prepareC: Accepts refData to have 0 or 2 number of alleles per loci. This allows references to have zero alleles (i.e. a missing marker).
  - Updated genDataset: References with empty loci are substituted with a random individual.
  - Updated deconvolve:
  -- Input parameter "unknownonly" is removed. The genotype of all contributors are now shown.
  -- outputs rankG=deconvlist,pG=pG are removed
  -- table1 is now changed: Includes combined geno prob per marker.
  -- table3 and table 4 added: Includes marginal geno and allele prob per marker per contributors
  - Updated simDOdistr: Allele-list Element in refData[[loc]][[ref]] can now be NULL or numeric() => Supports missing loci. These are substituted with a random individual.
• Bugs causing newest R versions (3.4.0) to give error is fixed.
  - We have replaced "<-glayout" with "<-gframe" at following lines in the code: Line 1407,Line 673,Line 2411,Line 2412,Line 2413,Line 2454,Line 2455,Line 2456.
• Bug in "View database": The number of loci compared to stains was following the same order as the individual elements in the database. This is now fixed.
  - Notice: Non-contributors assumes to have all markers (i.e. non-missing loci)
  - Missing markers (or not checked) is given "numeric()" as a substitute of the allele names, giving length=0. This is important for the prepareC function.

Updates in euroformix_0.6.3: (22.10.2016):

• In PP-plots: 0.05 and 0.05/n coverage envelopes are now superimposed to the PP-points (n is number of points). These are theoretical order-quantiles based on the uniform distribution. See EuroForMix manual v1.8 for more details.
• If single alleles are missing in a reference data file, the user is notified that there are missing alleles. This happens for some exports where homozygous alleles are only given ones.
• The kit.txt file has been updated: The kits "Fusion 6C", "PowerPlex16", "24plex", "ESSPlexPlus", "ESSPlexSEPlus", "ESSPlexSEQS", "YfilerPlus" have been added.
• Bug fixed when devonvolution table tries to show NA (thanks to Lourdes Prieto for pointing out this bug). For instance, this happened for empty evidence markers.
• Bug fixed when marker names include a space: check.names=FALSE has been added to read.table for preventing colNames changes. For instance, now "Penta D" is not changed to "Penta.D" such that it becomes unrecognizable to markers in "kit.txt".
• In "All results" (used to generate report):
  -> Bug fixed so that the contributor names become correct (thanks to Federica Alessandrini for pointing out the names become wrong when conditional references are not first in the reference list).
  -> The username ("User:") has been added to the file.
  -> "LR-per-marker" result has been added.
  -> Version name of euroformix R package is added.
  -> "(Based on xx MCMC samples)" is now set equal to the sample size in "Set number of samples" under "Toolbar->MCMC".
  -> Header in "All results" removed the phrase "of the maximum likelihood results"
• The fitted values of the sum peak height distribution are no longer printed (when clicking "View evidence").

Updates in euroformix_0.6.1: (01.09.2016):

• EuroForMix has been updated to v1.8 from v1.7.
• The following variables under "Model options" in "Model specification" has been moved to "File->Settings".
  -> Detection threshold
  -> fst-correction
  -> Probability of drop-in
  -> Drop-in peak height hyperparam (lambda)
  -> Stutter prior function.
• "Settings" were added to "File" toolbar:
  -> Easy Mode can be selected: Simplifies the MLE analysis by deactivating non-important buttons
  -> Maximum number of loci to visualize (maxloc) can be selected
  -> The setting values are stored and remembered (even when the program is closed).
• Information when clicking the saving button "All results" has been added: Now includes Version, Timestamp, Data, Model options, Hypotheses and LR values (MLE and 5% quantile from LR-sensitivity). Variables are also renamed.
• A popup-warning is given to the user when the maximum log-likelihood value is -Inf.
• Buttons in "Step 3)" under Import data has been rearranged.
• Removed warning in parallellization.
• Fixed bug in efm(): gtable does not accept NA values anymore. These are changed to "-".
• The calculation speed of exactMACdistr is now linear with number of markers.
• Button "Continuous LR (Bayesian based)" was removed from "MLE fit" tab.
• Update in contLikMCMC:
  logpX <- logdmvnorm(postth, mean = th0, cholC = chol(Sigma0))
  replaced by
  logpX <- logdmvnorm(postth, mean = th0, cholC = chol(delta*Sigma0))

Updates in euroformix_0.5.0: (14.01.2016)

• Parallell processing using snow R-package: Considered when number of unknowns are at least 3 (not implemented for non-contributor simulation and database searching yet). Number of processes will be equal number of random startpoints required in the optimization.
• The drop-out alleles from the references are no longer included when assuming the stutter-model. This implies that the LR results may slightly change for situations where reference alleles has dropped out.
• Bug found in Database search when 1) replicates are considered 2) locus in the population are not in same order as in the evidence. These are fixed.
• integrate() replaced by adaptIntegrate(rel.error=0.01) in validMLEmodel to increase robustness of integral calculations.
• simDOdistr() extended to handle a given drop-out probability for any of the contributors.
• When multiple evidence are checked:
  -> plotEPG now presents multiple evidence in one figure.
  -> Peak height sum per locus now presents multiple evidence in one figure. The model is now using all replicates simultanously.

Updates in euroformix_0.4.0: (19.10.2015)

• The marginal posterior probabilities for the top ranked genotype at each marker for each unknown contributors are shown as deconvolution layout "Marginal".
• Two result layouts are now available for deconvolution, "Joint" and "Marginal".
• EuroForMix has been updated to v1.7 from v1.6.

Updates in euroformix_0.3.11 (from 0.3.4): (17.10.2015)

• EuroForMix has been updated to v1.6 from v1.5.
• The posterior probabilities in the deconvolution are now also printed out per locus with corresponding genotype combinations.
• Using Bayesian based LR in database searching is now possible.
• Plotting peak heights with GlobalFiler and similar kits with more than 4 dyes is now possible. PlotEPG now uses the color directly from inst/extdata/kit.txt. Thanks to Inaki Yurrebaso for pointing out the error.
• Prints out the estimated Bayesian based LR when clicking 'LR Sensitivity' based on the MCMC simualations.
• The likelihood function (build in C++) does not restrict the mixture proportions of the unknown contributors to be decreasing anymore. The reason is that this restriction sometimes causes problem for the maximum likelihood approach.
• Replicates are now possible also for Bayesian based LR.
• Max evaluations in integration possible to adjust:
  -> 'Set maximum number of evaluations' has been added to Integration under the toolbar. By default this is value is unlimited.
• Improved startpoints used for optimizating the likelihood function found by doing regression on sum of the peak heights per markers.
  -> P.H.expectation, P.H.variability, Degrad.slope
• The degradation model was added to the genData-function.
• Symmetry in mixture proportions utilized also for more than 2 contributors:
  => Evaluating integration more accurate for:
  -> 3 contributors with 3 unknowns (1/6-size).
  -> 4 contributors with 4 unknowns (1/12-size).
  -> 4 contributors with 3 unknowns (1/2-size).
  -> 5 contributors with 5 unknowns (1/20-size).
  -> 5 contributors with 4 unknowns (1/3-size).
  -> 5 contributors with 3 unknowns (1/2-size).
  -> 6 contributors with 6 unknowns (1/25-size).
  -> 6 contributors with 5 unknowns (1/4-size).
  -> 6 contributors with 4 unknowns (1/3-size).
  -> 6 contributors with 3 unknowns (1/2-size).
• Bugs fixed:
  -> Bug in non-contribtor simulation for Bayesian based LR fixed
  -> Bug in deconvolution has been fixed: for the case of only one proposed genotype combination for any marker (typically for locus drop-out).
• Text-changes:
  "Integration based" -> "Bayesian based"