Elixir - testing a full script
up vote
0
down vote
favorite
I'm writing a test to check a function (called automatically by GenServer when a new file enters a folder) that calls other functions in the same module with pipes in order to read a file, process its content to insert it if needed and returns a list (:errors and :ok maps).
results looks like :
[
error: "Data not found",
ok: %MyModule{
field1: field1data,
field2: field2data
},
ok: %MyModule{
field1: field1data,
field2: field2data
},
error: "Data not found"
the code :
def processFile(file) do
insertResultsMap =
File.read!(file)
|> getLines()
|> extractMainData()
|> Enum.map(fn(x) -> insertLines(x) end)
|> Enum.group_by(fn x -> elem(x, 0) end)
handleErrors(Map.get(insertResultsMap, :error))
updateAnotherTableWithLines(Map.get(insertResultsMap, :ok))
end
defp getLines(docContent) do
String.split(docContent, "n")
end
defp extractMainData(docLines) do
Enum.map(fn(x) -> String.split(x, ",") end)
end
defp insertLines([field1, field2, field3, field4]) do
Attrs = %{
field1: String.trim(field1),
field2: String.trim(field2),
field3: String.trim(field3),
field4: String.trim(field4)
}
mymodule.create_stuff(Attrs)
end
defp handleErrors(errors) do
{:ok, file} = File.open(@errorsFile, [:append])
saveErrors(file, errors)
File.close(file)
end
defp saveErrors(_, ), do: :ok
defp saveErrors(file, [{:error, changeset}|rest]) do
changes = for {key, value} <- changeset.changes do
"#{key} #{value}"
end
errors = for {key, {message, _}} <- changeset.errors do
"#{key} #{message}"
end
errorData = "data: #{Enum.join(changes, ", ")} nErrors: #{Enum.join(errors, ", ")}nn"
IO.binwrite(file, errorData)
saveErrors(file, rest)
end
defp updateAnotherTableWithLines(insertedLines) do
Enum.map(insertedLines, fn {:ok, x} -> updateOtherTable(x) end)
end
defp updateOtherTable(dataForUpdate) do
"CLOSE" -> otherModule.doStuff(dataForUpdate.field1, dataForUpdate.field2)
end
I have several questions, and some will be pretty basic since I'm still learning :
- What do you think of the code ? Any advices ? (take into account I voluntarily obfuscated names).
- If I want to test this, is it the right way to test only
processFile
function ? Or should I make public more of them and test them individually ? - When I test the
processFile
function, I check that I'm receiving a list. Any way to make sure this list has only elements I'm waiting for, thuserror: "String"
orok: %{}
" ?
testing pattern-matching elixir phoenix-framework gen-server
add a comment |
up vote
0
down vote
favorite
I'm writing a test to check a function (called automatically by GenServer when a new file enters a folder) that calls other functions in the same module with pipes in order to read a file, process its content to insert it if needed and returns a list (:errors and :ok maps).
results looks like :
[
error: "Data not found",
ok: %MyModule{
field1: field1data,
field2: field2data
},
ok: %MyModule{
field1: field1data,
field2: field2data
},
error: "Data not found"
the code :
def processFile(file) do
insertResultsMap =
File.read!(file)
|> getLines()
|> extractMainData()
|> Enum.map(fn(x) -> insertLines(x) end)
|> Enum.group_by(fn x -> elem(x, 0) end)
handleErrors(Map.get(insertResultsMap, :error))
updateAnotherTableWithLines(Map.get(insertResultsMap, :ok))
end
defp getLines(docContent) do
String.split(docContent, "n")
end
defp extractMainData(docLines) do
Enum.map(fn(x) -> String.split(x, ",") end)
end
defp insertLines([field1, field2, field3, field4]) do
Attrs = %{
field1: String.trim(field1),
field2: String.trim(field2),
field3: String.trim(field3),
field4: String.trim(field4)
}
mymodule.create_stuff(Attrs)
end
defp handleErrors(errors) do
{:ok, file} = File.open(@errorsFile, [:append])
saveErrors(file, errors)
File.close(file)
end
defp saveErrors(_, ), do: :ok
defp saveErrors(file, [{:error, changeset}|rest]) do
changes = for {key, value} <- changeset.changes do
"#{key} #{value}"
end
errors = for {key, {message, _}} <- changeset.errors do
"#{key} #{message}"
end
errorData = "data: #{Enum.join(changes, ", ")} nErrors: #{Enum.join(errors, ", ")}nn"
IO.binwrite(file, errorData)
saveErrors(file, rest)
end
defp updateAnotherTableWithLines(insertedLines) do
Enum.map(insertedLines, fn {:ok, x} -> updateOtherTable(x) end)
end
defp updateOtherTable(dataForUpdate) do
"CLOSE" -> otherModule.doStuff(dataForUpdate.field1, dataForUpdate.field2)
end
I have several questions, and some will be pretty basic since I'm still learning :
- What do you think of the code ? Any advices ? (take into account I voluntarily obfuscated names).
- If I want to test this, is it the right way to test only
processFile
function ? Or should I make public more of them and test them individually ? - When I test the
processFile
function, I check that I'm receiving a list. Any way to make sure this list has only elements I'm waiting for, thuserror: "String"
orok: %{}
" ?
testing pattern-matching elixir phoenix-framework gen-server
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I'm writing a test to check a function (called automatically by GenServer when a new file enters a folder) that calls other functions in the same module with pipes in order to read a file, process its content to insert it if needed and returns a list (:errors and :ok maps).
results looks like :
[
error: "Data not found",
ok: %MyModule{
field1: field1data,
field2: field2data
},
ok: %MyModule{
field1: field1data,
field2: field2data
},
error: "Data not found"
the code :
def processFile(file) do
insertResultsMap =
File.read!(file)
|> getLines()
|> extractMainData()
|> Enum.map(fn(x) -> insertLines(x) end)
|> Enum.group_by(fn x -> elem(x, 0) end)
handleErrors(Map.get(insertResultsMap, :error))
updateAnotherTableWithLines(Map.get(insertResultsMap, :ok))
end
defp getLines(docContent) do
String.split(docContent, "n")
end
defp extractMainData(docLines) do
Enum.map(fn(x) -> String.split(x, ",") end)
end
defp insertLines([field1, field2, field3, field4]) do
Attrs = %{
field1: String.trim(field1),
field2: String.trim(field2),
field3: String.trim(field3),
field4: String.trim(field4)
}
mymodule.create_stuff(Attrs)
end
defp handleErrors(errors) do
{:ok, file} = File.open(@errorsFile, [:append])
saveErrors(file, errors)
File.close(file)
end
defp saveErrors(_, ), do: :ok
defp saveErrors(file, [{:error, changeset}|rest]) do
changes = for {key, value} <- changeset.changes do
"#{key} #{value}"
end
errors = for {key, {message, _}} <- changeset.errors do
"#{key} #{message}"
end
errorData = "data: #{Enum.join(changes, ", ")} nErrors: #{Enum.join(errors, ", ")}nn"
IO.binwrite(file, errorData)
saveErrors(file, rest)
end
defp updateAnotherTableWithLines(insertedLines) do
Enum.map(insertedLines, fn {:ok, x} -> updateOtherTable(x) end)
end
defp updateOtherTable(dataForUpdate) do
"CLOSE" -> otherModule.doStuff(dataForUpdate.field1, dataForUpdate.field2)
end
I have several questions, and some will be pretty basic since I'm still learning :
- What do you think of the code ? Any advices ? (take into account I voluntarily obfuscated names).
- If I want to test this, is it the right way to test only
processFile
function ? Or should I make public more of them and test them individually ? - When I test the
processFile
function, I check that I'm receiving a list. Any way to make sure this list has only elements I'm waiting for, thuserror: "String"
orok: %{}
" ?
testing pattern-matching elixir phoenix-framework gen-server
I'm writing a test to check a function (called automatically by GenServer when a new file enters a folder) that calls other functions in the same module with pipes in order to read a file, process its content to insert it if needed and returns a list (:errors and :ok maps).
results looks like :
[
error: "Data not found",
ok: %MyModule{
field1: field1data,
field2: field2data
},
ok: %MyModule{
field1: field1data,
field2: field2data
},
error: "Data not found"
the code :
def processFile(file) do
insertResultsMap =
File.read!(file)
|> getLines()
|> extractMainData()
|> Enum.map(fn(x) -> insertLines(x) end)
|> Enum.group_by(fn x -> elem(x, 0) end)
handleErrors(Map.get(insertResultsMap, :error))
updateAnotherTableWithLines(Map.get(insertResultsMap, :ok))
end
defp getLines(docContent) do
String.split(docContent, "n")
end
defp extractMainData(docLines) do
Enum.map(fn(x) -> String.split(x, ",") end)
end
defp insertLines([field1, field2, field3, field4]) do
Attrs = %{
field1: String.trim(field1),
field2: String.trim(field2),
field3: String.trim(field3),
field4: String.trim(field4)
}
mymodule.create_stuff(Attrs)
end
defp handleErrors(errors) do
{:ok, file} = File.open(@errorsFile, [:append])
saveErrors(file, errors)
File.close(file)
end
defp saveErrors(_, ), do: :ok
defp saveErrors(file, [{:error, changeset}|rest]) do
changes = for {key, value} <- changeset.changes do
"#{key} #{value}"
end
errors = for {key, {message, _}} <- changeset.errors do
"#{key} #{message}"
end
errorData = "data: #{Enum.join(changes, ", ")} nErrors: #{Enum.join(errors, ", ")}nn"
IO.binwrite(file, errorData)
saveErrors(file, rest)
end
defp updateAnotherTableWithLines(insertedLines) do
Enum.map(insertedLines, fn {:ok, x} -> updateOtherTable(x) end)
end
defp updateOtherTable(dataForUpdate) do
"CLOSE" -> otherModule.doStuff(dataForUpdate.field1, dataForUpdate.field2)
end
I have several questions, and some will be pretty basic since I'm still learning :
- What do you think of the code ? Any advices ? (take into account I voluntarily obfuscated names).
- If I want to test this, is it the right way to test only
processFile
function ? Or should I make public more of them and test them individually ? - When I test the
processFile
function, I check that I'm receiving a list. Any way to make sure this list has only elements I'm waiting for, thuserror: "String"
orok: %{}
" ?
testing pattern-matching elixir phoenix-framework gen-server
testing pattern-matching elixir phoenix-framework gen-server
asked Nov 19 at 15:16
Jeremy Belolo
1,14921943
1,14921943
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
up vote
1
down vote
accepted
What do you think of the code? Any advices? (take into account I voluntarily obfuscated names).
Opinion based.
If I want to test this, is it the right way to test only processFile function?
Yes.
Or should I make public more of them and test them individually?
No, this is an implementation detail and testing it is an anti-pattern.
When I test the processFile function, I check that I'm receiving a list. Any way to make sure this list has only elements I'm waiting for, thus error: "String" or ok: %{}"?
You receive a Keyword
. To check the explicit value, one might use:
foo = processFile(file)
assert not is_nil(foo[:ok])
OTOH, I’d better return a map from there and pattern match it:
assert %{ok: _} = processFile(file)
To assert that the result does not have anything save for :ok
s and :error
s, one might use list subtraction:
assert Enum.uniq(Keyword.keys(result)) -- [:ok, :error] ==
Hey, thanks for helping. Moving to a map would remove the duplicate keys, wouldn't it ? Also, I don't want to test that returnedkeyword
actually contains any:ok
tuple, but rather that it doesn't contain something unexpected - other that:ok
and:error
tuples.
– Jeremy Belolo
Nov 19 at 15:46
1
1. yes, if you need duplicate keys,Keyword
is the way to go. 2. Then simply assertKeyword.keys(result) -- ~w|ok error|a ==
.
– Aleksei Matiushkin
Nov 19 at 16:44
This won't work - as per doc def "Removes the first occurrence of an item on the left list for each item on the right" - only the first ! To remove all occurences of:ok
and:error
it won't work. Converting to aMapSet
would remove some elements... But then, I could convert to a map and bind to a new variable in order to check only expected elements' keys are here, without relying on this new variable for anything else :) Thanks again
– Jeremy Belolo
Nov 20 at 7:52
Ah, indeed.Enum.uniq/1
is required.
– Aleksei Matiushkin
Nov 20 at 11:31
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
1
down vote
accepted
What do you think of the code? Any advices? (take into account I voluntarily obfuscated names).
Opinion based.
If I want to test this, is it the right way to test only processFile function?
Yes.
Or should I make public more of them and test them individually?
No, this is an implementation detail and testing it is an anti-pattern.
When I test the processFile function, I check that I'm receiving a list. Any way to make sure this list has only elements I'm waiting for, thus error: "String" or ok: %{}"?
You receive a Keyword
. To check the explicit value, one might use:
foo = processFile(file)
assert not is_nil(foo[:ok])
OTOH, I’d better return a map from there and pattern match it:
assert %{ok: _} = processFile(file)
To assert that the result does not have anything save for :ok
s and :error
s, one might use list subtraction:
assert Enum.uniq(Keyword.keys(result)) -- [:ok, :error] ==
Hey, thanks for helping. Moving to a map would remove the duplicate keys, wouldn't it ? Also, I don't want to test that returnedkeyword
actually contains any:ok
tuple, but rather that it doesn't contain something unexpected - other that:ok
and:error
tuples.
– Jeremy Belolo
Nov 19 at 15:46
1
1. yes, if you need duplicate keys,Keyword
is the way to go. 2. Then simply assertKeyword.keys(result) -- ~w|ok error|a ==
.
– Aleksei Matiushkin
Nov 19 at 16:44
This won't work - as per doc def "Removes the first occurrence of an item on the left list for each item on the right" - only the first ! To remove all occurences of:ok
and:error
it won't work. Converting to aMapSet
would remove some elements... But then, I could convert to a map and bind to a new variable in order to check only expected elements' keys are here, without relying on this new variable for anything else :) Thanks again
– Jeremy Belolo
Nov 20 at 7:52
Ah, indeed.Enum.uniq/1
is required.
– Aleksei Matiushkin
Nov 20 at 11:31
add a comment |
up vote
1
down vote
accepted
What do you think of the code? Any advices? (take into account I voluntarily obfuscated names).
Opinion based.
If I want to test this, is it the right way to test only processFile function?
Yes.
Or should I make public more of them and test them individually?
No, this is an implementation detail and testing it is an anti-pattern.
When I test the processFile function, I check that I'm receiving a list. Any way to make sure this list has only elements I'm waiting for, thus error: "String" or ok: %{}"?
You receive a Keyword
. To check the explicit value, one might use:
foo = processFile(file)
assert not is_nil(foo[:ok])
OTOH, I’d better return a map from there and pattern match it:
assert %{ok: _} = processFile(file)
To assert that the result does not have anything save for :ok
s and :error
s, one might use list subtraction:
assert Enum.uniq(Keyword.keys(result)) -- [:ok, :error] ==
Hey, thanks for helping. Moving to a map would remove the duplicate keys, wouldn't it ? Also, I don't want to test that returnedkeyword
actually contains any:ok
tuple, but rather that it doesn't contain something unexpected - other that:ok
and:error
tuples.
– Jeremy Belolo
Nov 19 at 15:46
1
1. yes, if you need duplicate keys,Keyword
is the way to go. 2. Then simply assertKeyword.keys(result) -- ~w|ok error|a ==
.
– Aleksei Matiushkin
Nov 19 at 16:44
This won't work - as per doc def "Removes the first occurrence of an item on the left list for each item on the right" - only the first ! To remove all occurences of:ok
and:error
it won't work. Converting to aMapSet
would remove some elements... But then, I could convert to a map and bind to a new variable in order to check only expected elements' keys are here, without relying on this new variable for anything else :) Thanks again
– Jeremy Belolo
Nov 20 at 7:52
Ah, indeed.Enum.uniq/1
is required.
– Aleksei Matiushkin
Nov 20 at 11:31
add a comment |
up vote
1
down vote
accepted
up vote
1
down vote
accepted
What do you think of the code? Any advices? (take into account I voluntarily obfuscated names).
Opinion based.
If I want to test this, is it the right way to test only processFile function?
Yes.
Or should I make public more of them and test them individually?
No, this is an implementation detail and testing it is an anti-pattern.
When I test the processFile function, I check that I'm receiving a list. Any way to make sure this list has only elements I'm waiting for, thus error: "String" or ok: %{}"?
You receive a Keyword
. To check the explicit value, one might use:
foo = processFile(file)
assert not is_nil(foo[:ok])
OTOH, I’d better return a map from there and pattern match it:
assert %{ok: _} = processFile(file)
To assert that the result does not have anything save for :ok
s and :error
s, one might use list subtraction:
assert Enum.uniq(Keyword.keys(result)) -- [:ok, :error] ==
What do you think of the code? Any advices? (take into account I voluntarily obfuscated names).
Opinion based.
If I want to test this, is it the right way to test only processFile function?
Yes.
Or should I make public more of them and test them individually?
No, this is an implementation detail and testing it is an anti-pattern.
When I test the processFile function, I check that I'm receiving a list. Any way to make sure this list has only elements I'm waiting for, thus error: "String" or ok: %{}"?
You receive a Keyword
. To check the explicit value, one might use:
foo = processFile(file)
assert not is_nil(foo[:ok])
OTOH, I’d better return a map from there and pattern match it:
assert %{ok: _} = processFile(file)
To assert that the result does not have anything save for :ok
s and :error
s, one might use list subtraction:
assert Enum.uniq(Keyword.keys(result)) -- [:ok, :error] ==
edited Nov 20 at 11:31
answered Nov 19 at 15:37
Aleksei Matiushkin
77.4k95088
77.4k95088
Hey, thanks for helping. Moving to a map would remove the duplicate keys, wouldn't it ? Also, I don't want to test that returnedkeyword
actually contains any:ok
tuple, but rather that it doesn't contain something unexpected - other that:ok
and:error
tuples.
– Jeremy Belolo
Nov 19 at 15:46
1
1. yes, if you need duplicate keys,Keyword
is the way to go. 2. Then simply assertKeyword.keys(result) -- ~w|ok error|a ==
.
– Aleksei Matiushkin
Nov 19 at 16:44
This won't work - as per doc def "Removes the first occurrence of an item on the left list for each item on the right" - only the first ! To remove all occurences of:ok
and:error
it won't work. Converting to aMapSet
would remove some elements... But then, I could convert to a map and bind to a new variable in order to check only expected elements' keys are here, without relying on this new variable for anything else :) Thanks again
– Jeremy Belolo
Nov 20 at 7:52
Ah, indeed.Enum.uniq/1
is required.
– Aleksei Matiushkin
Nov 20 at 11:31
add a comment |
Hey, thanks for helping. Moving to a map would remove the duplicate keys, wouldn't it ? Also, I don't want to test that returnedkeyword
actually contains any:ok
tuple, but rather that it doesn't contain something unexpected - other that:ok
and:error
tuples.
– Jeremy Belolo
Nov 19 at 15:46
1
1. yes, if you need duplicate keys,Keyword
is the way to go. 2. Then simply assertKeyword.keys(result) -- ~w|ok error|a ==
.
– Aleksei Matiushkin
Nov 19 at 16:44
This won't work - as per doc def "Removes the first occurrence of an item on the left list for each item on the right" - only the first ! To remove all occurences of:ok
and:error
it won't work. Converting to aMapSet
would remove some elements... But then, I could convert to a map and bind to a new variable in order to check only expected elements' keys are here, without relying on this new variable for anything else :) Thanks again
– Jeremy Belolo
Nov 20 at 7:52
Ah, indeed.Enum.uniq/1
is required.
– Aleksei Matiushkin
Nov 20 at 11:31
Hey, thanks for helping. Moving to a map would remove the duplicate keys, wouldn't it ? Also, I don't want to test that returned
keyword
actually contains any :ok
tuple, but rather that it doesn't contain something unexpected - other that :ok
and :error
tuples.– Jeremy Belolo
Nov 19 at 15:46
Hey, thanks for helping. Moving to a map would remove the duplicate keys, wouldn't it ? Also, I don't want to test that returned
keyword
actually contains any :ok
tuple, but rather that it doesn't contain something unexpected - other that :ok
and :error
tuples.– Jeremy Belolo
Nov 19 at 15:46
1
1
1. yes, if you need duplicate keys,
Keyword
is the way to go. 2. Then simply assert Keyword.keys(result) -- ~w|ok error|a ==
.– Aleksei Matiushkin
Nov 19 at 16:44
1. yes, if you need duplicate keys,
Keyword
is the way to go. 2. Then simply assert Keyword.keys(result) -- ~w|ok error|a ==
.– Aleksei Matiushkin
Nov 19 at 16:44
This won't work - as per doc def "Removes the first occurrence of an item on the left list for each item on the right" - only the first ! To remove all occurences of
:ok
and :error
it won't work. Converting to a MapSet
would remove some elements... But then, I could convert to a map and bind to a new variable in order to check only expected elements' keys are here, without relying on this new variable for anything else :) Thanks again– Jeremy Belolo
Nov 20 at 7:52
This won't work - as per doc def "Removes the first occurrence of an item on the left list for each item on the right" - only the first ! To remove all occurences of
:ok
and :error
it won't work. Converting to a MapSet
would remove some elements... But then, I could convert to a map and bind to a new variable in order to check only expected elements' keys are here, without relying on this new variable for anything else :) Thanks again– Jeremy Belolo
Nov 20 at 7:52
Ah, indeed.
Enum.uniq/1
is required.– Aleksei Matiushkin
Nov 20 at 11:31
Ah, indeed.
Enum.uniq/1
is required.– Aleksei Matiushkin
Nov 20 at 11:31
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53377630%2felixir-testing-a-full-script%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown