Limit of loading dataset

[Bill Wang]

I am an octave newbie. I want to load a big data set saved as .csv
file. It is really big, about 50K examples. After loading, I checked
my variable size, the variable size is smaller the the data set. And I
think the data set is not fully loaded. Is there any way to solve it?
Or I should import to Access and export? Please give me a relative
detailed explanation.

[hugocoolens]

How did you load it?
like this?:
y=load('test.csv');
were there comment-lines in the beginning of your file?

hugo

    I am an octave newbie. I want to load a big data set saved as .csv
file. It is really big, about 50K examples. After loading, I checked
my variable size, the variable size is smaller the the data set. And I
think the data set is not fully loaded. Is there any way to solve it?
Or I should import to Access and export? Please give me a relative
detailed explanation.

[Bill Wang]

Post by hugocoolens
How did you load it?
 y=load('test.csv');

I loaded by csvread ('training.csv')

Post by hugocoolens
were there comment-lines in the beginning of your file?

Yes, there is one comment line at the beginning.
I wonder the format I used in this group is correct. New to Google
Group, somebody warned me about this.

Post by hugocoolens
hugo

    I am an octave newbie. I want to load a big data set saved as .csv
file. It is really big, about 50K examples. After loading, I checked
my variable size, the variable size is smaller the the data set. And I
think the data set is not fully loaded. Is there any way to solve it?
Or I should import to Access and export? Please give me a relative
detailed explanation.

[hugocoolens]

I did a little test with
a file called 'test.csv'
#test-file
1,1
2,4
3,9
4,16

as you can see the only difference between load en csvread is the
first line

octave:1> y=csvread('test.csv')
y =

0 0
1 1
2 4
3 9
4 16

octave:2> y2=load('test.csv')
y2 =

1 1
2 4
3 9
4 16

Post by Bill Wang

Post by hugocoolens
How did you load it?
 y=load('test.csv');

I loaded by csvread ('training.csv')

Post by hugocoolens
were there comment-lines in the beginning of your file?

Yes, there is one comment line at the beginning.
I wonder the format I used in this group is correct.  New to Google
Group, somebody warned me about this.

Post by hugocoolens
hugo

    I am an octave newbie. I want to load a big data set saved as .csv
file. It is really big, about 50K examples. After loading, I checked
my variable size, the variable size is smaller the the data set. And I
think the data set is not fully loaded. Is there any way to solve it?
Or I should import to Access and export? Please give me a relative
detailed explanation.

[Bill Wang]

I figured out how to load. I change the file type to txt. by rename
it. I finally loaded successfully. Bur during the process, I found out
you can not have data like data or null and header in any place. And
load command for txt is faster than csvread command.

Post by hugocoolens
I did a little test with
a file called 'test.csv'
#test-file
1,1
2,4
3,9
4,16
as you can see the only difference between load en csvread is the
first line
octave:1> y=csvread('test.csv')
y =
    0    0
    1    1
    2    4
    3    9
    4   16
octave:2> y2=load('test.csv')
y2 =
    1    1
    2    4
    3    9
    4   16

Post by Bill Wang

Post by hugocoolens
How did you load it?
 y=load('test.csv');

I loaded by csvread ('training.csv')

Post by hugocoolens
were there comment-lines in the beginning of your file?

Yes, there is one comment line at the beginning.
I wonder the format I used in this group is correct.  New to Google
Group, somebody warned me about this.

Post by hugocoolens
hugo

    I am an octave newbie. I want to load a big data set saved as .csv
file. It is really big, about 50K examples. After loading, I checked
my variable size, the variable size is smaller the the data set. And I
think the data set is not fully loaded. Is there any way to solve it?
Or I should import to Access and export? Please give me a relative
detailed explanation.