Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game. |
Notices |
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
Are you new to LinuxQuestions.org? Visit the following links:
Site Howto |
Site FAQ |
Sitemap |
Register Now
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
|
|
10-30-2024, 02:50 AM
|
#1
|
LQ Newbie
Registered: Sep 2018
Posts: 26
Rep:
|
bash script to curl send gz file with attachment via sendgrid
Sendgrid says to use:
Code:
curl --request POST \
--url https://api.sendgrid.com/v3/mail/send \
--header 'authorization: Bearer YOUR_API_KEY' \
--header 'Content-Type: application/json' \
--data '{"personalizations": [{"to": [{"email": "recipient@example.com"}]}],"from": {"email": "sender@example.com"},"subject":"Hello, World!","content": [{"type": "text/html","value": "Hey, Please find attachment."}], "attachments": [{"content": "BASE64_ENCODED_CONTENT", "type": "text/plain", "filename": "attachment.txt"}]}'
However will not encode anything larger than a few bytes. The gz file in question here that needs to be sent is ~4mb. What is a solution?
Last edited by primuspaul; 10-30-2024 at 11:28 AM.
|
|
|
10-30-2024, 03:11 AM
|
#2
|
LQ Addict
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 23,009
|
what do you mean by that?
Quote:
base64 will not encode anything larger than a few bytes
|
How did you try it?
|
|
1 members found this post helpful.
|
10-30-2024, 03:13 AM
|
#3
|
LQ Newbie
Registered: Sep 2018
Posts: 26
Original Poster
Rep:
|
Quote:
Originally Posted by pan64
what do you mean by that?
How did you try it?
|
In a bash script on a Debian headless server. It was a variable like base64 <filename.gz>. I get an error if it's more than 76 characters.
|
|
|
10-30-2024, 03:14 AM
|
#4
|
Senior Member
Registered: Jul 2020
Posts: 1,203
|
There is nothing wrong with base64 - I just used it to encode a 1.5G file, your issue is probably caused by the command line length limit. You can check it with 'getconf ARG_MAX' (also includes the environment size), on my machines it's 2M, so your data won't fit. Save it as a disk file (-d @file) or pipe it (-d -).
|
|
|
10-30-2024, 03:16 AM
|
#5
|
LQ Newbie
Registered: Sep 2018
Posts: 26
Original Poster
Rep:
|
Quote:
Originally Posted by lvm_
There is nothing wrong with base64 - I just used it to encode a 1.5G file, your issue is probably caused by the command line length limit. You can check it with 'getconf ARG_MAX' (also includes the environment size), on my machines it's 2M, so your data won't fit. Save it as a disk file (-d @file) or pipe it (-d -).
|
You're right. How would the piping look in a bash script? Not extremely familiar with bash.
|
|
|
10-30-2024, 03:18 AM
|
#6
|
Senior Member
Registered: Jul 2020
Posts: 1,203
|
Quote:
Originally Posted by primuspaul
I get an error if it's more than 76 characters.
|
76 is the line size for base64, apparently your script just doesn't handle line breaks properly
|
|
|
10-30-2024, 03:21 AM
|
#7
|
LQ Addict
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 23,009
|
Quote:
Originally Posted by primuspaul
In a bash script on a Debian headless server. It was a variable like base64 <filename.gz>. I get an error if it's more than 76 characters.
|
this is not a correct answer. Would be nice to see exactly what did you try.
Otherwise base64 has a man page and you can find a lot of examples/tutorials on the net (about it and about a lot of other useful things).
The basic usage is simple: base64 file > encoded_file.txt.
So please post your script an we will try to fix it together.
Obviously please remove sensitive information before posting.
|
|
|
10-30-2024, 03:35 AM
|
#8
|
LQ Newbie
Registered: Sep 2018
Posts: 26
Original Poster
Rep:
|
Quote:
Originally Posted by pan64
this is not a correct answer. Would be nice to see exactly what did you try.
Otherwise base64 has a man page and you can find a lot of examples/tutorials on the net (about it and about a lot of other useful things).
The basic usage is simple: base64 file > encoded_file.txt.
So please post your script an we will try to fix it together.
Obviously please remove sensitive information before posting.
|
Code:
#!/bin/bash
base64 < yeast.gz > yeast.txt
BASE64=$(cat /home/yeast.txt)
#echo $BASE64
curl --request POST \
--url https://api.sendgrid.com/v3/mail/send \
--header "Authorization: Bearer SG.ggg" \
--header 'Content-Type: application/json' \
--data '{"personalizations": [{"to": [{"email": "me@me.com"}]}],"from": {"email": "me@me.com"},"subject": "Send" ,"content": [{"type": "text/plain", "value": "and"}], "attachments": [{"content": "'$BASE64'", "type": "text/plain", "filename": "yeast.txt"}]}'
I get:
Code:
Argument list too long
|
|
|
10-30-2024, 03:43 AM
|
#9
|
LQ Addict
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 23,009
|
yes, the variable BASE64 is long, you cannot put it into a command like that.
All that content should be put into a file (like data.json) and you can use --data "@data.json"
The question is now how can you create that data.json.
As a simple solution you can try something like this:
Code:
{
echo '{"personalizations": [{"to": [{"email": "me@me.com"}]}],"from": {"email": "me@me.com"},"subject": "Send" ,"content": [{"type": "text/plain", "value": "and"}], "attachments": [{"content": "'
base64 < yeast.gz
echo '", "type": "text/plain", "filename": "yeast.txt"}]}'
} > data.json
|
|
|
10-30-2024, 03:51 AM
|
#10
|
LQ Newbie
Registered: Sep 2018
Posts: 26
Original Poster
Rep:
|
Quote:
Originally Posted by pan64
yes, the variable BASE64 is long, you cannot put it into a command like that.
All that content should be put into a file (like data.json) and you can use --data "@data.json"
The question is now how can you create that data.json.
As a simple solution you can try something like this:
Code:
{
echo '{"personalizations": [{"to": [{"email": "me@me.com"}]}],"from": {"email": "me@me.com"},"subject": "Send" ,"content": [{"type": "text/plain", "value": "and"}], "attachments": [{"content": "'
base64 < yeast.gz
echo '", "type": "text/plain", "filename": "yeast.txt"}]}'
} > data.json
|
Code is now:
Code:
#!/bin/bash
base64 < yeast.gz > yeast.txt
BASE64=$(cat /home/yeast.txt)
curl --request POST \
--url https://api.sendgrid.com/v3/mail/send \
--header "Authorization: Bearer SG.GA" \
--header 'Content-Type: application/json' \
--data '{"personalizations": [{"to": [{"email": "me@me.com"}]}],"from": {"email": "me@me.com"},"subject": "Send"
,"content": [{"type": "text/plain", "value": "and"}], "attachments": [{"content": "'base64 < /home/yeast.gz echo'", "type": "text/plain",
"filename": "yeast.txt"}]}'
I get:
Code:
{"errors":[{"message":"Bad Request","field":null,"help":null}]}curl: (3) unmatched close brace/bracket in URL position 53:
echo", "type": "text/plain", "filename": "yeast.txt"}]}
^
|
|
|
10-30-2024, 03:54 AM
|
#11
|
LQ Addict
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 23,009
|
no, I did not tell that.
Code:
#!/bin/bash
{
echo '{"personalizations": [{"to": [{"email": "me@me.com"}]}],"from": {"email": "me@me.com"},"subject": "Send" ,"content": [{"type": "text/plain", "value": "and"}], "attachments": [{"content": "'
base64 < yeast.gz
echo '", "type": "text/plain", "filename": "yeast.txt"}]}'
} > data.json
curl --request POST \
--url https://api.sendgrid.com/v3/mail/send \
--header "Authorization: Bearer SG.GA" \
--header 'Content-Type: application/json' \
--data "@data.json"
|
|
1 members found this post helpful.
|
10-30-2024, 03:56 AM
|
#12
|
Senior Member
Registered: Oct 2011
Location: Budapest
Distribution: Debian/GNU/Linux, AIX
Posts: 4,935
|
TL;DR don't put file contents into variables, only file names.
|
|
|
10-30-2024, 04:06 AM
|
#13
|
LQ Newbie
Registered: Sep 2018
Posts: 26
Original Poster
Rep:
|
Quote:
Originally Posted by pan64
no, I did not tell that.
Code:
#!/bin/bash
{
echo '{"personalizations": [{"to": [{"email": "me@me.com"}]}],"from": {"email": "me@me.com"},"subject": "Send" ,"content": [{"type": "text/plain", "value": "and"}], "attachments": [{"content": "'
base64 < yeast.gz
echo '", "type": "text/plain", "filename": "yeast.txt"}]}'
} > data.json
curl --request POST \
--url https://api.sendgrid.com/v3/mail/send \
--header "Authorization: Bearer SG.GA" \
--header 'Content-Type: application/json' \
--data "@data.json"
|
This appears to work, thanks. Just wondering though the file seems to go over as the gz file? So I can even send it as a .gz attachment as it does not seem to require any steps to convert besides just unzipping just like the original file. Works even better than I thought it would.
|
|
|
10-30-2024, 04:11 AM
|
#14
|
LQ Addict
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 23,009
|
I don't really understand, base64 will generate another file which is usually bigger than the original file and contains only a special set of characters.
Using this script you can encode a .gz file (with base64) and send it.
Decoding will happen on the receiver side (where this encoded file will be used to restore the original content).
If your problem is solved please mark the thread as solved.
Also if you want to say thanks just click on yes.
|
|
1 members found this post helpful.
|
All times are GMT -5. The time now is 09:09 PM.
|
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
|
Latest Threads
LQ News
|
|