What is 'NaN' in the third column in retreived txt files

[jiangshan529]

Hello, I have retrieved the information from a .hic file. However, I found 'nan' values in the third column. What does this mean? Is this missing values? Why should such value appear? Thanks!

60000 60000 3709.8407084423666
60000 65000 nan
60000 70000 nan
60000 85000 974.9826322352375
85000 85000 6662.110695460637

[sa501428]

If a particular column was very sparse and KR/SCALE had to discard the column/row when normalizing the matrix, then all entries in the discarded row/column will be NaN.

[moshe-olshansky]

It means that these rows (bins) had to be removed during normalization. You should not see any NaN (Not a Number) in raw (un-normalized) map.

[jiangshan529]

It means that these rows (bins) had to be removed during normalization. You should not see any NaN (Not a Number) in raw (un-normalized) map.

Hi, thanks for your reply. I have run Straw with normalization set to 'None'. And I am not sure this time why there are float numbers in the third column.

60000 60000 1
60000 65000 1
60000 70000 1
60000 85000 1
85000 85000 26
85000 90000 16
90000 90000 53
90000 95000 12
95000 95000 27
58570000 58605000 67.67821603676141
58575000 58605000 179.20274301772898
58580000 58605000 200.11589450569903
58585000 58605000 100.45844430432754
58590000 58605000 237.57597248105807
58595000 58605000 600.7486450401082
58605000 58605000 7401.773853029534

[moshe-olshansky]

Are you using oe (observed over expected)? If so, float numbers should not surprise you. On Tuesday, 15 November 2022 at 02:40:34 pm AEDT, jiangshan529 ***@***.***> wrote: It means that these rows (bins) had to be removed during normalization. You should not see any NaN (Not a Number) in raw (un-normalized) map. Hi, thanks for your reply. I have run Straw with normalization set to 'None'. And I am not sure this time why there are float numbers in the third column. 60000 60000 1 60000 65000 1 60000 70000 1 60000 85000 1 85000 85000 26 85000 90000 16 90000 90000 53 90000 95000 12 95000 95000 27 58570000 58605000 67.67821603676141 58575000 58605000 179.20274301772898 58580000 58605000 200.11589450569903 58585000 58605000 100.45844430432754 58590000 58605000 237.57597248105807 58595000 58605000 600.7486450401082 58605000 58605000 7401.773853029534 — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>

[jiangshan529]

Are you using oe (observed over expected)? If so, float numbers should not surprise you. On Tuesday, 15 November 2022 at 02:40:34 pm AEDT, jiangshan529 @.> wrote: It means that these rows (bins) had to be removed during normalization. You should not see any NaN (Not a Number) in raw (un-normalized) map. Hi, thanks for your reply. I have run Straw with normalization set to 'None'. And I am not sure this time why there are float numbers in the third column. 60000 60000 1 60000 65000 1 60000 70000 1 60000 85000 1 85000 85000 26 85000 90000 16 90000 90000 53 90000 95000 12 95000 95000 27 58570000 58605000 67.67821603676141 58575000 58605000 179.20274301772898 58580000 58605000 200.11589450569903 58585000 58605000 100.45844430432754 58590000 58605000 237.57597248105807 58595000 58605000 600.7486450401082 58605000 58605000 7401.773853029534 — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.>

Hi, the code I am using is 'result = straw.straw('NONE',"./4DNFI2TK7L2F.hic", "19", "19", "BP", 5000, 'chr19_5k.txt')'

[moshe-olshansky]

Where does this hic file come from? Is it a (weighted) combination of several maps? On Tuesday, 15 November 2022 at 02:53:12 pm AEDT, jiangshan529 ***@***.***> wrote: Are you using oe (observed over expected)? If so, float numbers should not surprise you. On Tuesday, 15 November 2022 at 02:40:34 pm AEDT, jiangshan529 @.> wrote: It means that these rows (bins) had to be removed during normalization. You should not see any NaN (Not a Number) in raw (un-normalized) map. Hi, thanks for your reply. I have run Straw with normalization set to 'None'. And I am not sure this time why there are float numbers in the third column. 60000 60000 1 60000 65000 1 60000 70000 1 60000 85000 1 85000 85000 26 85000 90000 16 90000 90000 53 90000 95000 12 95000 95000 27 58570000 58605000 67.67821603676141 58575000 58605000 179.20274301772898 58580000 58605000 200.11589450569903 58585000 58605000 100.45844430432754 58590000 58605000 237.57597248105807 58595000 58605000 600.7486450401082 58605000 58605000 7401.773853029534 — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.> Hi, the code I am using is 'result = straw.straw('NONE',"./4DNFI2TK7L2F.hic", "19", "19", "BP", 5000, 'chr19_5k.txt')' — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>

[jiangshan529]

Where does this hic file come from? Is it a (weighted) combination of several maps? On Tuesday, 15 November 2022 at 02:53:12 pm AEDT, jiangshan529 @.> wrote: Are you using oe (observed over expected)? If so, float numbers should not surprise you. On Tuesday, 15 November 2022 at 02:40:34 pm AEDT, jiangshan529 @.> wrote: It means that these rows (bins) had to be removed during normalization. You should not see any NaN (Not a Number) in raw (un-normalized) map. Hi, thanks for your reply. I have run Straw with normalization set to 'None'. And I am not sure this time why there are float numbers in the third column. 60000 60000 1 60000 65000 1 60000 70000 1 60000 85000 1 85000 85000 26 85000 90000 16 90000 90000 53 90000 95000 12 95000 95000 27 58570000 58605000 67.67821603676141 58575000 58605000 179.20274301772898 58580000 58605000 200.11589450569903 58585000 58605000 100.45844430432754 58590000 58605000 237.57597248105807 58595000 58605000 600.7486450401082 58605000 58605000 7401.773853029534 — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.> Hi, the code I am using is 'result = straw.straw('NONE',"./4DNFI2TK7L2F.hic", "19", "19", "BP", 5000, 'chr19_5k.txt')' — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.>

It is from the 4D genome project, performed by micro-C method.
https://data.4dnucleome.org/files-processed/4DNFI2TK7L2F/#details

[moshe-olshansky]

I think that you should check how exactly this hic file was created. Maybe read carefully or ask the person in charge. On Tuesday, 15 November 2022 at 03:02:28 pm AEDT, jiangshan529 ***@***.***> wrote: Where does this hic file come from? Is it a (weighted) combination of several maps? On Tuesday, 15 November 2022 at 02:53:12 pm AEDT, jiangshan529 @.> wrote: Are you using oe (observed over expected)? If so, float numbers should not surprise you. On Tuesday, 15 November 2022 at 02:40:34 pm AEDT, jiangshan529 @.> wrote: It means that these rows (bins) had to be removed during normalization. You should not see any NaN (Not a Number) in raw (un-normalized) map. Hi, thanks for your reply. I have run Straw with normalization set to 'None'. And I am not sure this time why there are float numbers in the third column. 60000 60000 1 60000 65000 1 60000 70000 1 60000 85000 1 85000 85000 26 85000 90000 16 90000 90000 53 90000 95000 12 95000 95000 27 58570000 58605000 67.67821603676141 58575000 58605000 179.20274301772898 58580000 58605000 200.11589450569903 58585000 58605000 100.45844430432754 58590000 58605000 237.57597248105807 58595000 58605000 600.7486450401082 58605000 58605000 7401.773853029534 — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.> Hi, the code I am using is 'result = straw.straw('NONE',"./4DNFI2TK7L2F.hic", "19", "19", "BP", 5000, 'chr19_5k.txt')' — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.> It is from the 4D genome project, performed by micro-C method. https://data.4dnucleome.org/files-processed/4DNFI2TK7L2F/#details — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>

[moshe-olshansky]

By the way, have you tried using dump command in juicer tools? Does it produce identical results (to straw)? On Tuesday, 15 November 2022 at 03:02:28 pm AEDT, jiangshan529 ***@***.***> wrote: Where does this hic file come from? Is it a (weighted) combination of several maps? On Tuesday, 15 November 2022 at 02:53:12 pm AEDT, jiangshan529 @.> wrote: Are you using oe (observed over expected)? If so, float numbers should not surprise you. On Tuesday, 15 November 2022 at 02:40:34 pm AEDT, jiangshan529 @.> wrote: It means that these rows (bins) had to be removed during normalization. You should not see any NaN (Not a Number) in raw (un-normalized) map. Hi, thanks for your reply. I have run Straw with normalization set to 'None'. And I am not sure this time why there are float numbers in the third column. 60000 60000 1 60000 65000 1 60000 70000 1 60000 85000 1 85000 85000 26 85000 90000 16 90000 90000 53 90000 95000 12 95000 95000 27 58570000 58605000 67.67821603676141 58575000 58605000 179.20274301772898 58580000 58605000 200.11589450569903 58585000 58605000 100.45844430432754 58590000 58605000 237.57597248105807 58595000 58605000 600.7486450401082 58605000 58605000 7401.773853029534 — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.> Hi, the code I am using is 'result = straw.straw('NONE',"./4DNFI2TK7L2F.hic", "19", "19", "BP", 5000, 'chr19_5k.txt')' — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.> It is from the 4D genome project, performed by micro-C method. https://data.4dnucleome.org/files-processed/4DNFI2TK7L2F/#details — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>

[jiangshan529]

By the way, have you tried using dump command in juicer tools? Does it produce identical results (to straw)? On Tuesday, 15 November 2022 at 03:02:28 pm AEDT, jiangshan529 @.> wrote: Where does this hic file come from? Is it a (weighted) combination of several maps? On Tuesday, 15 November 2022 at 02:53:12 pm AEDT, jiangshan529 @.> wrote: Are you using oe (observed over expected)? If so, float numbers should not surprise you. On Tuesday, 15 November 2022 at 02:40:34 pm AEDT, jiangshan529 @.> wrote: It means that these rows (bins) had to be removed during normalization. You should not see any NaN (Not a Number) in raw (un-normalized) map. Hi, thanks for your reply. I have run Straw with normalization set to 'None'. And I am not sure this time why there are float numbers in the third column. 60000 60000 1 60000 65000 1 60000 70000 1 60000 85000 1 85000 85000 26 85000 90000 16 90000 90000 53 90000 95000 12 95000 95000 27 58570000 58605000 67.67821603676141 58575000 58605000 179.20274301772898 58580000 58605000 200.11589450569903 58585000 58605000 100.45844430432754 58590000 58605000 237.57597248105807 58595000 58605000 600.7486450401082 58605000 58605000 7401.773853029534 — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.> Hi, the code I am using is 'result = straw.straw('NONE',"./4DNFI2TK7L2F.hic", "19", "19", "BP", 5000, 'chr19_5k.txt')' — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.> It is from the 4D genome project, performed by micro-C method. https://data.4dnucleome.org/files-processed/4DNFI2TK7L2F/#details — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.>

Sorry, I just downloaded the .hic file. How should use the dump command?

[sa501428]

Please post this question to the forum. We reserve github issues for bugs.

[sa501428]

https://groups.google.com/g/3d-genomics
And that way, the community as a whole will also benefit from the answers.
Thanks!

[moshe-olshansky]

Have you downloaded juicer_tools.jar?If so, do java -jar juicer_tools.jar or/and java -jar juicer_tools.jar dump to see the usage. On Tuesday, 15 November 2022 at 03:20:17 pm AEDT, jiangshan529 ***@***.***> wrote: By the way, have you tried using dump command in juicer tools? Does it produce identical results (to straw)? On Tuesday, 15 November 2022 at 03:02:28 pm AEDT, jiangshan529 @.> wrote: Where does this hic file come from? Is it a (weighted) combination of several maps? On Tuesday, 15 November 2022 at 02:53:12 pm AEDT, jiangshan529 @.> wrote: Are you using oe (observed over expected)? If so, float numbers should not surprise you. On Tuesday, 15 November 2022 at 02:40:34 pm AEDT, jiangshan529 @.> wrote: It means that these rows (bins) had to be removed during normalization. You should not see any NaN (Not a Number) in raw (un-normalized) map. Hi, thanks for your reply. I have run Straw with normalization set to 'None'. And I am not sure this time why there are float numbers in the third column. 60000 60000 1 60000 65000 1 60000 70000 1 60000 85000 1 85000 85000 26 85000 90000 16 90000 90000 53 90000 95000 12 95000 95000 27 58570000 58605000 67.67821603676141 58575000 58605000 179.20274301772898 58580000 58605000 200.11589450569903 58585000 58605000 100.45844430432754 58590000 58605000 237.57597248105807 58595000 58605000 600.7486450401082 58605000 58605000 7401.773853029534 — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.> Hi, the code I am using is 'result = straw.straw('NONE',"./4DNFI2TK7L2F.hic", "19", "19", "BP", 5000, 'chr19_5k.txt')' — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.> It is from the 4D genome project, performed by micro-C method. https://data.4dnucleome.org/files-processed/4DNFI2TK7L2F/#details — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.> Sorry, I just downloaded the .hic file. How should use the dump command? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>

[jiangshan529]

Have you downloaded juicer_tools.jar?If so, do java -jar juicer_tools.jar or/and java -jar juicer_tools.jar dump to see the usage. On Tuesday, 15 November 2022 at 03:20:17 pm AEDT, jiangshan529 @.> wrote: By the way, have you tried using dump command in juicer tools? Does it produce identical results (to straw)? On Tuesday, 15 November 2022 at 03:02:28 pm AEDT, jiangshan529 @.> wrote: Where does this hic file come from? Is it a (weighted) combination of several maps? On Tuesday, 15 November 2022 at 02:53:12 pm AEDT, jiangshan529 @.> wrote: Are you using oe (observed over expected)? If so, float numbers should not surprise you. On Tuesday, 15 November 2022 at 02:40:34 pm AEDT, jiangshan529 @.> wrote: It means that these rows (bins) had to be removed during normalization. You should not see any NaN (Not a Number) in raw (un-normalized) map. Hi, thanks for your reply. I have run Straw with normalization set to 'None'. And I am not sure this time why there are float numbers in the third column. 60000 60000 1 60000 65000 1 60000 70000 1 60000 85000 1 85000 85000 26 85000 90000 16 90000 90000 53 90000 95000 12 95000 95000 27 58570000 58605000 67.67821603676141 58575000 58605000 179.20274301772898 58580000 58605000 200.11589450569903 58585000 58605000 100.45844430432754 58590000 58605000 237.57597248105807 58595000 58605000 600.7486450401082 58605000 58605000 7401.773853029534 — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.> Hi, the code I am using is 'result = straw.straw('NONE',"./4DNFI2TK7L2F.hic", "19", "19", "BP", 5000, 'chr19_5k.txt')' — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.> It is from the 4D genome project, performed by micro-C method. https://data.4dnucleome.org/files-processed/4DNFI2TK7L2F/#details — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.> Sorry, I just downloaded the .hic file. How should use the dump command? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.>

Hi, I have run it.

When I use 'observed', no float numbers appeared.

when I use 'oe', it looks like this:
60000 60000 3.0029617E-4
60000 65000 9.408559E-4
60000 70000 0.0040292614
60000 85000 0.012957309
85000 85000 0.0078077004
85000 90000 0.015053694
90000 90000 0.015915697
90000 95000 0.011290271
95000 95000 0.008107997