sed to fix a messy nginx.conf
$begingroup$
I have a few servers to fix that repeatedly get their nginx.conf file messed up due to a rogue deploy script.
These are the three sed commands I'm using to fix at present:
sed -i '' 's/^([[:blank:]]*)#([[:blank:]]*)server_names_hash_bucket_size/1server_names_hash_bucket_size/g' nginx.conf
The first finds any kind of blank space at the start of a line followed by a pound symbol and the string server_names_hash_bucket_size
. Then it replaces it with the original blank space (to honour any indentations) and the string without the pound.
sed -i '' '/^([[:blank:]]*)server_names_hash_bucket_size/s/[0-9]//g' nginx.conf
The second gets rid of any numbers:
sed -i '' '/^([[:blank:]]*)server_names_hash_bucket_size/s/;/64;/g' nginx.conf
The third finds the semi colon and replaces it with the necessary value 64;
I'm currently testing this in terminal OSX, but ultimately will be used on Linux machines.
Does anyone know how to tidy this up and can point out any potential Linux gotchas with sed?
regex linux sed
New contributor
$endgroup$
add a comment |
$begingroup$
I have a few servers to fix that repeatedly get their nginx.conf file messed up due to a rogue deploy script.
These are the three sed commands I'm using to fix at present:
sed -i '' 's/^([[:blank:]]*)#([[:blank:]]*)server_names_hash_bucket_size/1server_names_hash_bucket_size/g' nginx.conf
The first finds any kind of blank space at the start of a line followed by a pound symbol and the string server_names_hash_bucket_size
. Then it replaces it with the original blank space (to honour any indentations) and the string without the pound.
sed -i '' '/^([[:blank:]]*)server_names_hash_bucket_size/s/[0-9]//g' nginx.conf
The second gets rid of any numbers:
sed -i '' '/^([[:blank:]]*)server_names_hash_bucket_size/s/;/64;/g' nginx.conf
The third finds the semi colon and replaces it with the necessary value 64;
I'm currently testing this in terminal OSX, but ultimately will be used on Linux machines.
Does anyone know how to tidy this up and can point out any potential Linux gotchas with sed?
regex linux sed
New contributor
$endgroup$
add a comment |
$begingroup$
I have a few servers to fix that repeatedly get their nginx.conf file messed up due to a rogue deploy script.
These are the three sed commands I'm using to fix at present:
sed -i '' 's/^([[:blank:]]*)#([[:blank:]]*)server_names_hash_bucket_size/1server_names_hash_bucket_size/g' nginx.conf
The first finds any kind of blank space at the start of a line followed by a pound symbol and the string server_names_hash_bucket_size
. Then it replaces it with the original blank space (to honour any indentations) and the string without the pound.
sed -i '' '/^([[:blank:]]*)server_names_hash_bucket_size/s/[0-9]//g' nginx.conf
The second gets rid of any numbers:
sed -i '' '/^([[:blank:]]*)server_names_hash_bucket_size/s/;/64;/g' nginx.conf
The third finds the semi colon and replaces it with the necessary value 64;
I'm currently testing this in terminal OSX, but ultimately will be used on Linux machines.
Does anyone know how to tidy this up and can point out any potential Linux gotchas with sed?
regex linux sed
New contributor
$endgroup$
I have a few servers to fix that repeatedly get their nginx.conf file messed up due to a rogue deploy script.
These are the three sed commands I'm using to fix at present:
sed -i '' 's/^([[:blank:]]*)#([[:blank:]]*)server_names_hash_bucket_size/1server_names_hash_bucket_size/g' nginx.conf
The first finds any kind of blank space at the start of a line followed by a pound symbol and the string server_names_hash_bucket_size
. Then it replaces it with the original blank space (to honour any indentations) and the string without the pound.
sed -i '' '/^([[:blank:]]*)server_names_hash_bucket_size/s/[0-9]//g' nginx.conf
The second gets rid of any numbers:
sed -i '' '/^([[:blank:]]*)server_names_hash_bucket_size/s/;/64;/g' nginx.conf
The third finds the semi colon and replaces it with the necessary value 64;
I'm currently testing this in terminal OSX, but ultimately will be used on Linux machines.
Does anyone know how to tidy this up and can point out any potential Linux gotchas with sed?
regex linux sed
regex linux sed
New contributor
New contributor
edited 5 mins ago
Jamal♦
30.3k11119227
30.3k11119227
New contributor
asked yesterday
mar627mar627
235
235
New contributor
New contributor
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
In the general case, the answer to any question regarding sed is "use something else."
Not that sed is not useful! It definitely is! It's just that it's best for simple tasks that have compact, obvious solutions.
In particular, Perl is good at replacing sed scripts, thanks to its more expressive regular expressions and uncomplicated access to conventional programming constructs. It even comes with an automated tool to do the translation from sed to Perl, called s2p
.
perl -pli.bak -e'
next unless s/^(s*)(#s*)?(?=server_names_hash_bucket_size)/$1/;
tr/d//d;
s/;/64;/;
' nginx.conf
The newlines are optional and included for readability. Line by line:
-p
means "print each line after running this code against it." This should be familiar since sed works exactly the same way.
-l
means "remove newlines before running the code and reinstate them before printing."
-i.bak
means "edit in-place and save the original asoriginal.bak
".
-e
means the same thing as in sed.
- We're only interested in lines with
server_names_hash_bucket_size
, so skip to the next line if the replace fails (the unmodified line still prints; we're skipping the rest of this code but not the printing). - We need to include lines that had no
#
, so make the#
match optional with?
after(#s*)
.
(?=...)
is a lookahead; the text must exist but is not included in the left side of the substitution, so we don't need to include it on the right side. Only the (optional) leading whitespace ($1
) needs to be reinstated on the RHS.
- We're only interested in lines with
tr
works likey
in sed. This line is equivalent tos/[0-9]//g
(as far as we're concerned. Technically, there are digits other than 0-9 but they aren't going to appear in your nginx.conf.)Same as sed.
You can simplify even further by recognizing that the only significant parts of a server_names_hash_bucket_size
line, after you throw away the number, are the leading whitespace and terminal comments.
Preserve the indent as before, throw away as many non-comment characters as possible ([^#]*
) and rewrite the line in one replace operation:
perl -pli.bak -e' s/^(s*)(?:#s*)?(server_names_hash_bucket_size)[^#]*/$1$2 64;/ ' nginx.conf
(?:
makes that group non-capturing, so the second capture ($2
) will be server_names_hash_bucket_size
.
At this point, we're back in "simple enough for sed" territory:
s/^([[:blank:]]*)(#[[:blank:]]*)*(server_names_hash_bucket_size)[^#]*/13 64;/
$endgroup$
1
$begingroup$
Fantastic write up @oh-my-goodness, I'm going to adopt some of Perls more intuitive syntax for complex adjustments. The final 'simple enough for sed' solution works perfectly too; commented matches of the string are ignored correctly, indentations are honoured, and the 64 value is successfully placed where needed. Thank you
$endgroup$
– mar627
yesterday
1
$begingroup$
thanks! Note I made a small fix to the sed after your comment, to allow matching more than one space after#
.
$endgroup$
– Oh My Goodness
19 hours ago
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["\$", "\$"]]);
});
});
}, "mathjax-editing");
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "196"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
mar627 is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fcodereview.stackexchange.com%2fquestions%2f214508%2fsed-to-fix-a-messy-nginx-conf%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
In the general case, the answer to any question regarding sed is "use something else."
Not that sed is not useful! It definitely is! It's just that it's best for simple tasks that have compact, obvious solutions.
In particular, Perl is good at replacing sed scripts, thanks to its more expressive regular expressions and uncomplicated access to conventional programming constructs. It even comes with an automated tool to do the translation from sed to Perl, called s2p
.
perl -pli.bak -e'
next unless s/^(s*)(#s*)?(?=server_names_hash_bucket_size)/$1/;
tr/d//d;
s/;/64;/;
' nginx.conf
The newlines are optional and included for readability. Line by line:
-p
means "print each line after running this code against it." This should be familiar since sed works exactly the same way.
-l
means "remove newlines before running the code and reinstate them before printing."
-i.bak
means "edit in-place and save the original asoriginal.bak
".
-e
means the same thing as in sed.
- We're only interested in lines with
server_names_hash_bucket_size
, so skip to the next line if the replace fails (the unmodified line still prints; we're skipping the rest of this code but not the printing). - We need to include lines that had no
#
, so make the#
match optional with?
after(#s*)
.
(?=...)
is a lookahead; the text must exist but is not included in the left side of the substitution, so we don't need to include it on the right side. Only the (optional) leading whitespace ($1
) needs to be reinstated on the RHS.
- We're only interested in lines with
tr
works likey
in sed. This line is equivalent tos/[0-9]//g
(as far as we're concerned. Technically, there are digits other than 0-9 but they aren't going to appear in your nginx.conf.)Same as sed.
You can simplify even further by recognizing that the only significant parts of a server_names_hash_bucket_size
line, after you throw away the number, are the leading whitespace and terminal comments.
Preserve the indent as before, throw away as many non-comment characters as possible ([^#]*
) and rewrite the line in one replace operation:
perl -pli.bak -e' s/^(s*)(?:#s*)?(server_names_hash_bucket_size)[^#]*/$1$2 64;/ ' nginx.conf
(?:
makes that group non-capturing, so the second capture ($2
) will be server_names_hash_bucket_size
.
At this point, we're back in "simple enough for sed" territory:
s/^([[:blank:]]*)(#[[:blank:]]*)*(server_names_hash_bucket_size)[^#]*/13 64;/
$endgroup$
1
$begingroup$
Fantastic write up @oh-my-goodness, I'm going to adopt some of Perls more intuitive syntax for complex adjustments. The final 'simple enough for sed' solution works perfectly too; commented matches of the string are ignored correctly, indentations are honoured, and the 64 value is successfully placed where needed. Thank you
$endgroup$
– mar627
yesterday
1
$begingroup$
thanks! Note I made a small fix to the sed after your comment, to allow matching more than one space after#
.
$endgroup$
– Oh My Goodness
19 hours ago
add a comment |
$begingroup$
In the general case, the answer to any question regarding sed is "use something else."
Not that sed is not useful! It definitely is! It's just that it's best for simple tasks that have compact, obvious solutions.
In particular, Perl is good at replacing sed scripts, thanks to its more expressive regular expressions and uncomplicated access to conventional programming constructs. It even comes with an automated tool to do the translation from sed to Perl, called s2p
.
perl -pli.bak -e'
next unless s/^(s*)(#s*)?(?=server_names_hash_bucket_size)/$1/;
tr/d//d;
s/;/64;/;
' nginx.conf
The newlines are optional and included for readability. Line by line:
-p
means "print each line after running this code against it." This should be familiar since sed works exactly the same way.
-l
means "remove newlines before running the code and reinstate them before printing."
-i.bak
means "edit in-place and save the original asoriginal.bak
".
-e
means the same thing as in sed.
- We're only interested in lines with
server_names_hash_bucket_size
, so skip to the next line if the replace fails (the unmodified line still prints; we're skipping the rest of this code but not the printing). - We need to include lines that had no
#
, so make the#
match optional with?
after(#s*)
.
(?=...)
is a lookahead; the text must exist but is not included in the left side of the substitution, so we don't need to include it on the right side. Only the (optional) leading whitespace ($1
) needs to be reinstated on the RHS.
- We're only interested in lines with
tr
works likey
in sed. This line is equivalent tos/[0-9]//g
(as far as we're concerned. Technically, there are digits other than 0-9 but they aren't going to appear in your nginx.conf.)Same as sed.
You can simplify even further by recognizing that the only significant parts of a server_names_hash_bucket_size
line, after you throw away the number, are the leading whitespace and terminal comments.
Preserve the indent as before, throw away as many non-comment characters as possible ([^#]*
) and rewrite the line in one replace operation:
perl -pli.bak -e' s/^(s*)(?:#s*)?(server_names_hash_bucket_size)[^#]*/$1$2 64;/ ' nginx.conf
(?:
makes that group non-capturing, so the second capture ($2
) will be server_names_hash_bucket_size
.
At this point, we're back in "simple enough for sed" territory:
s/^([[:blank:]]*)(#[[:blank:]]*)*(server_names_hash_bucket_size)[^#]*/13 64;/
$endgroup$
1
$begingroup$
Fantastic write up @oh-my-goodness, I'm going to adopt some of Perls more intuitive syntax for complex adjustments. The final 'simple enough for sed' solution works perfectly too; commented matches of the string are ignored correctly, indentations are honoured, and the 64 value is successfully placed where needed. Thank you
$endgroup$
– mar627
yesterday
1
$begingroup$
thanks! Note I made a small fix to the sed after your comment, to allow matching more than one space after#
.
$endgroup$
– Oh My Goodness
19 hours ago
add a comment |
$begingroup$
In the general case, the answer to any question regarding sed is "use something else."
Not that sed is not useful! It definitely is! It's just that it's best for simple tasks that have compact, obvious solutions.
In particular, Perl is good at replacing sed scripts, thanks to its more expressive regular expressions and uncomplicated access to conventional programming constructs. It even comes with an automated tool to do the translation from sed to Perl, called s2p
.
perl -pli.bak -e'
next unless s/^(s*)(#s*)?(?=server_names_hash_bucket_size)/$1/;
tr/d//d;
s/;/64;/;
' nginx.conf
The newlines are optional and included for readability. Line by line:
-p
means "print each line after running this code against it." This should be familiar since sed works exactly the same way.
-l
means "remove newlines before running the code and reinstate them before printing."
-i.bak
means "edit in-place and save the original asoriginal.bak
".
-e
means the same thing as in sed.
- We're only interested in lines with
server_names_hash_bucket_size
, so skip to the next line if the replace fails (the unmodified line still prints; we're skipping the rest of this code but not the printing). - We need to include lines that had no
#
, so make the#
match optional with?
after(#s*)
.
(?=...)
is a lookahead; the text must exist but is not included in the left side of the substitution, so we don't need to include it on the right side. Only the (optional) leading whitespace ($1
) needs to be reinstated on the RHS.
- We're only interested in lines with
tr
works likey
in sed. This line is equivalent tos/[0-9]//g
(as far as we're concerned. Technically, there are digits other than 0-9 but they aren't going to appear in your nginx.conf.)Same as sed.
You can simplify even further by recognizing that the only significant parts of a server_names_hash_bucket_size
line, after you throw away the number, are the leading whitespace and terminal comments.
Preserve the indent as before, throw away as many non-comment characters as possible ([^#]*
) and rewrite the line in one replace operation:
perl -pli.bak -e' s/^(s*)(?:#s*)?(server_names_hash_bucket_size)[^#]*/$1$2 64;/ ' nginx.conf
(?:
makes that group non-capturing, so the second capture ($2
) will be server_names_hash_bucket_size
.
At this point, we're back in "simple enough for sed" territory:
s/^([[:blank:]]*)(#[[:blank:]]*)*(server_names_hash_bucket_size)[^#]*/13 64;/
$endgroup$
In the general case, the answer to any question regarding sed is "use something else."
Not that sed is not useful! It definitely is! It's just that it's best for simple tasks that have compact, obvious solutions.
In particular, Perl is good at replacing sed scripts, thanks to its more expressive regular expressions and uncomplicated access to conventional programming constructs. It even comes with an automated tool to do the translation from sed to Perl, called s2p
.
perl -pli.bak -e'
next unless s/^(s*)(#s*)?(?=server_names_hash_bucket_size)/$1/;
tr/d//d;
s/;/64;/;
' nginx.conf
The newlines are optional and included for readability. Line by line:
-p
means "print each line after running this code against it." This should be familiar since sed works exactly the same way.
-l
means "remove newlines before running the code and reinstate them before printing."
-i.bak
means "edit in-place and save the original asoriginal.bak
".
-e
means the same thing as in sed.
- We're only interested in lines with
server_names_hash_bucket_size
, so skip to the next line if the replace fails (the unmodified line still prints; we're skipping the rest of this code but not the printing). - We need to include lines that had no
#
, so make the#
match optional with?
after(#s*)
.
(?=...)
is a lookahead; the text must exist but is not included in the left side of the substitution, so we don't need to include it on the right side. Only the (optional) leading whitespace ($1
) needs to be reinstated on the RHS.
- We're only interested in lines with
tr
works likey
in sed. This line is equivalent tos/[0-9]//g
(as far as we're concerned. Technically, there are digits other than 0-9 but they aren't going to appear in your nginx.conf.)Same as sed.
You can simplify even further by recognizing that the only significant parts of a server_names_hash_bucket_size
line, after you throw away the number, are the leading whitespace and terminal comments.
Preserve the indent as before, throw away as many non-comment characters as possible ([^#]*
) and rewrite the line in one replace operation:
perl -pli.bak -e' s/^(s*)(?:#s*)?(server_names_hash_bucket_size)[^#]*/$1$2 64;/ ' nginx.conf
(?:
makes that group non-capturing, so the second capture ($2
) will be server_names_hash_bucket_size
.
At this point, we're back in "simple enough for sed" territory:
s/^([[:blank:]]*)(#[[:blank:]]*)*(server_names_hash_bucket_size)[^#]*/13 64;/
edited 23 hours ago
answered yesterday
Oh My GoodnessOh My Goodness
69029
69029
1
$begingroup$
Fantastic write up @oh-my-goodness, I'm going to adopt some of Perls more intuitive syntax for complex adjustments. The final 'simple enough for sed' solution works perfectly too; commented matches of the string are ignored correctly, indentations are honoured, and the 64 value is successfully placed where needed. Thank you
$endgroup$
– mar627
yesterday
1
$begingroup$
thanks! Note I made a small fix to the sed after your comment, to allow matching more than one space after#
.
$endgroup$
– Oh My Goodness
19 hours ago
add a comment |
1
$begingroup$
Fantastic write up @oh-my-goodness, I'm going to adopt some of Perls more intuitive syntax for complex adjustments. The final 'simple enough for sed' solution works perfectly too; commented matches of the string are ignored correctly, indentations are honoured, and the 64 value is successfully placed where needed. Thank you
$endgroup$
– mar627
yesterday
1
$begingroup$
thanks! Note I made a small fix to the sed after your comment, to allow matching more than one space after#
.
$endgroup$
– Oh My Goodness
19 hours ago
1
1
$begingroup$
Fantastic write up @oh-my-goodness, I'm going to adopt some of Perls more intuitive syntax for complex adjustments. The final 'simple enough for sed' solution works perfectly too; commented matches of the string are ignored correctly, indentations are honoured, and the 64 value is successfully placed where needed. Thank you
$endgroup$
– mar627
yesterday
$begingroup$
Fantastic write up @oh-my-goodness, I'm going to adopt some of Perls more intuitive syntax for complex adjustments. The final 'simple enough for sed' solution works perfectly too; commented matches of the string are ignored correctly, indentations are honoured, and the 64 value is successfully placed where needed. Thank you
$endgroup$
– mar627
yesterday
1
1
$begingroup$
thanks! Note I made a small fix to the sed after your comment, to allow matching more than one space after
#
.$endgroup$
– Oh My Goodness
19 hours ago
$begingroup$
thanks! Note I made a small fix to the sed after your comment, to allow matching more than one space after
#
.$endgroup$
– Oh My Goodness
19 hours ago
add a comment |
mar627 is a new contributor. Be nice, and check out our Code of Conduct.
mar627 is a new contributor. Be nice, and check out our Code of Conduct.
mar627 is a new contributor. Be nice, and check out our Code of Conduct.
mar627 is a new contributor. Be nice, and check out our Code of Conduct.
Thanks for contributing an answer to Code Review Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fcodereview.stackexchange.com%2fquestions%2f214508%2fsed-to-fix-a-messy-nginx-conf%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown