LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 10-30-2024, 02:50 AM   #1
primuspaul
LQ Newbie
 
Registered: Sep 2018
Posts: 26

Rep: Reputation: Disabled
bash script to curl send gz file with attachment via sendgrid


Sendgrid says to use:

Code:
curl --request POST \
  --url https://api.sendgrid.com/v3/mail/send \
  --header 'authorization: Bearer YOUR_API_KEY' \
  --header 'Content-Type: application/json' \
  --data '{"personalizations": [{"to": [{"email": "recipient@example.com"}]}],"from": {"email": "sender@example.com"},"subject":"Hello, World!","content": [{"type": "text/html","value": "Hey, Please find attachment."}], "attachments": [{"content": "BASE64_ENCODED_CONTENT", "type": "text/plain", "filename": "attachment.txt"}]}'
However
Code:
base64
will not encode anything larger than a few bytes. The gz file in question here that needs to be sent is ~4mb. What is a solution?

Last edited by primuspaul; 10-30-2024 at 11:28 AM.
 
Old 10-30-2024, 03:11 AM   #2
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 23,009

Rep: Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627
what do you mean by that?
Quote:
base64 will not encode anything larger than a few bytes
How did you try it?
 
1 members found this post helpful.
Old 10-30-2024, 03:13 AM   #3
primuspaul
LQ Newbie
 
Registered: Sep 2018
Posts: 26

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by pan64 View Post
what do you mean by that?

How did you try it?
In a bash script on a Debian headless server. It was a variable like base64 <filename.gz>. I get an error if it's more than 76 characters.
 
Old 10-30-2024, 03:14 AM   #4
lvm_
Senior Member
 
Registered: Jul 2020
Posts: 1,203

Rep: Reputation: 403Reputation: 403Reputation: 403Reputation: 403Reputation: 403
There is nothing wrong with base64 - I just used it to encode a 1.5G file, your issue is probably caused by the command line length limit. You can check it with 'getconf ARG_MAX' (also includes the environment size), on my machines it's 2M, so your data won't fit. Save it as a disk file (-d @file) or pipe it (-d -).
 
Old 10-30-2024, 03:16 AM   #5
primuspaul
LQ Newbie
 
Registered: Sep 2018
Posts: 26

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by lvm_ View Post
There is nothing wrong with base64 - I just used it to encode a 1.5G file, your issue is probably caused by the command line length limit. You can check it with 'getconf ARG_MAX' (also includes the environment size), on my machines it's 2M, so your data won't fit. Save it as a disk file (-d @file) or pipe it (-d -).
You're right. How would the piping look in a bash script? Not extremely familiar with bash.
 
Old 10-30-2024, 03:18 AM   #6
lvm_
Senior Member
 
Registered: Jul 2020
Posts: 1,203

Rep: Reputation: 403Reputation: 403Reputation: 403Reputation: 403Reputation: 403
Quote:
Originally Posted by primuspaul View Post
I get an error if it's more than 76 characters.
76 is the line size for base64, apparently your script just doesn't handle line breaks properly
 
Old 10-30-2024, 03:21 AM   #7
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 23,009

Rep: Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627
Quote:
Originally Posted by primuspaul View Post
In a bash script on a Debian headless server. It was a variable like base64 <filename.gz>. I get an error if it's more than 76 characters.
this is not a correct answer. Would be nice to see exactly what did you try.
Otherwise base64 has a man page and you can find a lot of examples/tutorials on the net (about it and about a lot of other useful things).
The basic usage is simple: base64 file > encoded_file.txt.
So please post your script an we will try to fix it together.
Obviously please remove sensitive information before posting.
 
Old 10-30-2024, 03:35 AM   #8
primuspaul
LQ Newbie
 
Registered: Sep 2018
Posts: 26

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by pan64 View Post
this is not a correct answer. Would be nice to see exactly what did you try.
Otherwise base64 has a man page and you can find a lot of examples/tutorials on the net (about it and about a lot of other useful things).
The basic usage is simple: base64 file > encoded_file.txt.
So please post your script an we will try to fix it together.
Obviously please remove sensitive information before posting.
Code:
#!/bin/bash

base64 < yeast.gz > yeast.txt
BASE64=$(cat /home/yeast.txt)
#echo $BASE64

curl --request POST \
  --url https://api.sendgrid.com/v3/mail/send \
  --header "Authorization: Bearer SG.ggg" \
  --header 'Content-Type: application/json' \
  --data '{"personalizations": [{"to": [{"email": "me@me.com"}]}],"from": {"email": "me@me.com"},"subject": "Send" ,"content": [{"type": "text/plain", "value": "and"}], "attachments": [{"content": "'$BASE64'", "type": "text/plain", "filename": "yeast.txt"}]}'
I get:
Code:
Argument list too long
 
Old 10-30-2024, 03:43 AM   #9
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 23,009

Rep: Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627
yes, the variable BASE64 is long, you cannot put it into a command like that.
All that content should be put into a file (like data.json) and you can use --data "@data.json"
The question is now how can you create that data.json.
As a simple solution you can try something like this:
Code:
{
echo '{"personalizations": [{"to": [{"email": "me@me.com"}]}],"from": {"email": "me@me.com"},"subject": "Send" ,"content": [{"type": "text/plain", "value": "and"}], "attachments": [{"content": "'
base64 < yeast.gz
echo '", "type": "text/plain", "filename": "yeast.txt"}]}'
} > data.json
 
Old 10-30-2024, 03:51 AM   #10
primuspaul
LQ Newbie
 
Registered: Sep 2018
Posts: 26

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by pan64 View Post
yes, the variable BASE64 is long, you cannot put it into a command like that.
All that content should be put into a file (like data.json) and you can use --data "@data.json"
The question is now how can you create that data.json.
As a simple solution you can try something like this:
Code:
{
echo '{"personalizations": [{"to": [{"email": "me@me.com"}]}],"from": {"email": "me@me.com"},"subject": "Send" ,"content": [{"type": "text/plain", "value": "and"}], "attachments": [{"content": "'
base64 < yeast.gz
echo '", "type": "text/plain", "filename": "yeast.txt"}]}'
} > data.json
Code is now:

Code:
#!/bin/bash

base64 < yeast.gz > yeast.txt
BASE64=$(cat /home/yeast.txt)

curl --request POST \
  --url https://api.sendgrid.com/v3/mail/send \
  --header "Authorization: Bearer SG.GA" \
  --header 'Content-Type: application/json' \
  --data '{"personalizations": [{"to": [{"email": "me@me.com"}]}],"from": {"email": "me@me.com"},"subject": "Send"
,"content": [{"type": "text/plain", "value": "and"}], "attachments": [{"content": "'base64 < /home/yeast.gz echo'", "type": "text/plain",
 "filename": "yeast.txt"}]}'
I get:

Code:
{"errors":[{"message":"Bad Request","field":null,"help":null}]}curl: (3) unmatched close brace/bracket in URL position 53:
echo", "type": "text/plain", "filename": "yeast.txt"}]}
                                                    ^
 
Old 10-30-2024, 03:54 AM   #11
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 23,009

Rep: Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627
no, I did not tell that.
Code:
#!/bin/bash

{
   echo '{"personalizations": [{"to": [{"email": "me@me.com"}]}],"from": {"email": "me@me.com"},"subject": "Send" ,"content": [{"type": "text/plain", "value": "and"}], "attachments": [{"content": "'
   base64 < yeast.gz
   echo '", "type": "text/plain", "filename": "yeast.txt"}]}'
} > data.json

curl --request POST \
  --url https://api.sendgrid.com/v3/mail/send \
  --header "Authorization: Bearer SG.GA" \
  --header 'Content-Type: application/json' \
  --data "@data.json"
 
1 members found this post helpful.
Old 10-30-2024, 03:56 AM   #12
NevemTeve
Senior Member
 
Registered: Oct 2011
Location: Budapest
Distribution: Debian/GNU/Linux, AIX
Posts: 4,935
Blog Entries: 1

Rep: Reputation: 1893Reputation: 1893Reputation: 1893Reputation: 1893Reputation: 1893Reputation: 1893Reputation: 1893Reputation: 1893Reputation: 1893Reputation: 1893Reputation: 1893
TL;DR don't put file contents into variables, only file names.
 
Old 10-30-2024, 04:06 AM   #13
primuspaul
LQ Newbie
 
Registered: Sep 2018
Posts: 26

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by pan64 View Post
no, I did not tell that.
Code:
#!/bin/bash

{
   echo '{"personalizations": [{"to": [{"email": "me@me.com"}]}],"from": {"email": "me@me.com"},"subject": "Send" ,"content": [{"type": "text/plain", "value": "and"}], "attachments": [{"content": "'
   base64 < yeast.gz
   echo '", "type": "text/plain", "filename": "yeast.txt"}]}'
} > data.json

curl --request POST \
  --url https://api.sendgrid.com/v3/mail/send \
  --header "Authorization: Bearer SG.GA" \
  --header 'Content-Type: application/json' \
  --data "@data.json"
This appears to work, thanks. Just wondering though the file seems to go over as the gz file? So I can even send it as a .gz attachment as it does not seem to require any steps to convert besides just unzipping just like the original file. Works even better than I thought it would.
 
Old 10-30-2024, 04:11 AM   #14
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 23,009

Rep: Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627Reputation: 7627
I don't really understand, base64 will generate another file which is usually bigger than the original file and contains only a special set of characters.
Using this script you can encode a .gz file (with base64) and send it.
Decoding will happen on the receiver side (where this encoded file will be used to restore the original content).

If your problem is solved please mark the thread as solved.
Also if you want to say thanks just click on yes.
 
1 members found this post helpful.
  


Reply

Tags
bash, curl, debian, sendgrid


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] need help w/script using variable to send attachment via mutt thealmightyos Linux - Software 1 09-10-2014 07:18 PM
Can't send file attachment with curl patapout Linux - Newbie 0 08-10-2012 05:34 PM
convert email files with attachment to save as attachment file (metamail?) ted_chou12 Linux - Software 2 04-08-2011 09:01 PM
cURL: Server has many IPs, how would I make a cURL script use those IPs to send data? guest Programming 0 04-11-2009 12:42 PM
howto send a mail with attachment via perl script ? cccc Programming 24 03-05-2004 08:49 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 09:09 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration