Download files from https server using username and password

url = 'https://cddis.nasa.gov/archive/gnss/products/0971/';

In this https server, I need to download "cod09710.eph.Z" file for a specific location in my computer. How I can download this file using username and password related to this https server?

Answer 1

[Rik]

You should be able to use websave. You can provide a username and password in weboptions:

weboptions

ans = 
  weboptions with properties:

      CharacterEncoding: 'auto'
              UserAgent: 'MATLAB 9.10.0.1716447 (R2021a) Update 4'
                Timeout: 5
               Username: ''
               Password: ''
                KeyName: ''
               KeyValue: ''
            ContentType: 'auto'
          ContentReader: []
              MediaType: 'auto'
          RequestMethod: 'auto'
            ArrayFormat: 'csv'
           HeaderFields: []
    CertificateFilename: 'default'

[sermet OGUTCU]

Dear @Rik, I created the weboptions as follows:

options = weboptions('Username','name','Password','mypassword');

After defining username and passw, could you show the next step?

[Rik]

I haven't tried myself, but this should work:

filename='';
url='';
name='name';
mypassword='mypassword';
options = weboptions('Username',name,'Password',mypassword);
websave(filename,url,options)

[sermet OGUTCU]

Dear @Rik, after I execute the codes, the html file of "cod09710.eph.Z" is downloaded (cod09710.eph.Z.html) instead of the original file.

[Rik]

It is difficult to diagnose, as I don't have your login details (nor should you share them).

You could try two things:

1. Use movefile to change the extension (this assumes this is actually the correct file, just the wrong extension).
2. Set 'ContentType' to 'raw' in weboptions.

Let me know if either works.

[sermet OGUTCU]

Dear @Rik, I added the filename I needed to download to the url, then the html problem was solved.

[Andre Ferreira]

Dear @sermet OGUTCU,

I am also trying to download some products from the CDDIS with Matlab but I cannot succeed.

Can you please provide more details on how you have done it?

Best regards,

Andre

[sermet OGUTCU]

Dear @Andre,

I use GAMP II open source software for downloading from the CDDIS. You can run GAMP II independently or using MATLAB.

[Patrik Brigant]

@Andre Ferreira Hi, are you using GAMP II aswell or did you solved the issue using Matlab? Thank you :)

Answer 2

[Yongjian Feng]

Try this:

username = 'Your username here';
password = 'Your password here';
authStr = [username ':' password];
base64Str = ['Basic ' matlab.net.base64encode(authStr)];
headers = {'Authorization': base64Str};
options = weboptions('HeaderFields',headers);
webread('http://ip.jsontest.com', options)

[sermet OGUTCU]

Dear Yongjian, I got the following warning and error after creating of headers:

headers = {'Authorization': base64Str}; == Warning: Colon operands must be real scalars.

options = weboptions('HeaderFields',headers); == Error using weboptions (line 352)

Expected HeaderFields to be of size Mx2, but it is of size 1x1.

[Yongjian Feng]

Try this instead:

username = 'Your username here';
password = 'Your password here';
options = weboptions('Username',username,'Password',password);
webread('http://ip.jsontest.com', options)

[sermet OGUTCU]

Dear Yongjian, the codes worked without error but it produces 1 x 79854 char variable. It did not download the file ("cod09710.eph.Z).

[Yongjian Feng]

Use websave to save it to a file

username = 'Your username here';
password = 'Your password here';
url = 'https://cddis.nasa.gov/archive/gnss/products/0971/';
localfilename = 'Local file to save to';
options = weboptions('Username',username,'Password',password);
websave(localfilename, url, options)

[Yongjian Feng]

It will be easier to debug this process using curl in a terminal outside matlab. But

1. What has been saved into your local drive? code09710.eph.Z.html? Is it in HTML format? You can go to a terminal and cd to the folder then more code09710.eph.Z.html
2. If it is a html, then the wrong file has been downloaded. If it is not a html file, change its extension to .Z and try to unzip it. If you can unzip it, most likely it is the one you want.

[sermet OGUTCU]

Dear Yongjian, thank you for your further explanations.

[weiyong yi]

Doesn't work on my PC at all. Do you have any further ideas?

thank you.