Updates to submission process?
-
- Posts: 183
- Joined: 31 Jan 2010 04:51
Updates to submission process?
I wonder if we could have a News link on the main page: I'm never quite sure whether I'm using the current AddWork form or not: http://www.cpdl.org/wiki/index.php/Form_talk:Add_work
2 unheralded and welcome developments I've noticed just lately are 1) the add edition tab on workpages fills in some work-related blanks in the form automatically, & 2) the file size and number of pages are now filled in when uploading!
2 of my longtime gripes are 1) editor details aren't yet filled in for logged-in users (I leave a filled out add work form in an open tab routinely, but sooner or later there are problems) & 2) it irks me to get an error message because I didn't type my email twice, at the beginning and at the end of the form. Since, as I understand it, the last is provided for non-editor uploaders, shouldn't it be an optional "if different from above" field?
2 unheralded and welcome developments I've noticed just lately are 1) the add edition tab on workpages fills in some work-related blanks in the form automatically, & 2) the file size and number of pages are now filled in when uploading!
2 of my longtime gripes are 1) editor details aren't yet filled in for logged-in users (I leave a filled out add work form in an open tab routinely, but sooner or later there are problems) & 2) it irks me to get an error message because I didn't type my email twice, at the beginning and at the end of the form. Since, as I understand it, the last is provided for non-editor uploaders, shouldn't it be an optional "if different from above" field?
-
- Posts: 53
- Joined: 13 Jul 2016 10:47
Re: Updates to submission process?
I confess that I am annoyed every time that I have to type in my e-mail address at all as a registered user, even though it is stored in the user data.
Could somebody describe in the help pages what I have to do to make this work? I think I use the new add-work-process. I press the upload buttons and get new tabs to upload, but must then enter file size and number of pages in the original tab manually.the file size and number of pages are now filled in when uploading!
Re: Updates to submission process?
The automated add score process was introduced in October 2013. It was announced using the main announcement banner, still available on archive.org (click to enlarge):
I have to admit that maintaining both processes available is a little bit cumbersome and sub-optimum, because, to keep the two processes consistent, some limitations of the traditional process tend to propagate to the automatic process. Given the very small number of contributions made with the traditional process nowadays, maybe it's time to remove the traditional process and get rid of its inherent limitations.
The above announcement was permanently shown for about one year, until October 2014. However, at that time the "Add scores to CPDL" function shown on the navigation panel on the left side of each page was still linking to the "Score submission guide" describing the traditional process, as both processes were (and still are) available to all users. The "Add scores to CPDL" function was eventually re-directed to the page describing the automatic process in November 2016. Contributors gradually migrated from the traditional submission process to the automated process. In 2018, only about 125 submissions out of about 4400 (i.e., approximately 2.8%) have been made using the traditional process.I have to admit that maintaining both processes available is a little bit cumbersome and sub-optimum, because, to keep the two processes consistent, some limitations of the traditional process tend to propagate to the automatic process. Given the very small number of contributions made with the traditional process nowadays, maybe it's time to remove the traditional process and get rid of its inherent limitations.
This is intentional because many submissions are made by another user and not by the editor. I'd suggest to activate the browser function that remembers the values entered previously in the same field, so that you get the field completed right away by your browser.Richard Mix wrote:editor details aren't yet filled in for logged-in users
Yes, this is possible, although this increases the probability that we don't get the contributor's email address if he/she forgets entering it. This is not a big problem with the automated process as the contributor is anyway identified through the username, while it was a real problem with the traditional process as there were no user identification features, the email address(es) entered were the only ways to identify the contributor or the editor. The automatic process has been designed to mimic the traditional process, so it inherited this specific characteristic.Richard Mix wrote:Since, as I understand it, the last is provided for non-editor uploaders, shouldn't it be an optional "if different from above" field?
See above. This is inherited from the traditional process that had no other ways to identify the contributor.Nikolaus Hold wrote:I confess that I am annoyed every time that I have to type in my e-mail address at all as a registered user, even though it is stored in the user data.
Please double check that you upload the pdf file using the "Upload now" button relevant to the box "Sheet music location (give URL, or name of file)". That specific button activates the calculation of the file size and of the number of pages. They are determined after the uploading is successfully completed, so the relevant fields are automatically calculated within few seconds after the uploading is done.Nikolaus Hold wrote:Could somebody describe in the help pages what I have to do to make this work? I think I use the new add-work-process. I press the upload buttons and get new tabs to upload, but must then enter file size and number of pages in the original tab manually.
Re: Updates to submission process?
I learned awhile ago (after a few misfires) that, with the automated process, one should NOT UPLOAD files BEFORE using the automated process ... but, INSTEAD, upload files using the UPLOAD NOW option within the automated process.
I hope this helps.
I hope this helps.
Charles H. Giffen
CPDL Board of Directors Chair
Admin at & Manager of ChoralWiki
CPDL Board of Directors Chair
Admin at & Manager of ChoralWiki
-
- Posts: 53
- Joined: 13 Jul 2016 10:47
Re: Updates to submission process?
I'm sorry, but I seem to be a particularly slow-witted person. I used the "Upload now" buttons every time (there are no others). Last time it worked like this:
So, no Scoreinfo and no music notation file. What did I do wrong?
- 1. I clicked on the first/top "Upload now" button and uploaded the pdf file in the new tab. Then I manually took over the URL (right mouse button -> copy link address, then Ctrl-v in the upload tab).
- 2. Since the page number and size of the PDF file should be automatically filled in, I did not enter it this time.
- 3. Then I clicked on the 4th/lowest "Upload now" button and uploaded the mxl file in the additional new tab. This time, I did not manually enter the URL in the upload tab to test whether it will be automatically filled in.
- 4. After filling the rest of the fields, I made the attached hardcopy and then pressed "Add works data".
Code: Select all
{{#Legend:}}
*{{PostedDate|2018-12-25}} {{CPDLno|52710}} [[Media:Nh_Schubert_geistlich_Abendlied.pdf|{{pdf}}]]
{{Editor|NIkolaus Hold|2018-12-25}}{{ScoreInfo|Unknown||}}{{Copy|Creative Commons Attribution Non-Commercial}}
:'''Edition notes:''' Arr. from [[Deutscher Sängerhain (Ernst Schmidt)]]. {{MXL}}
Re: Updates to submission process?
That's strange. The data of a certain file (e.g., file size and number of a pages in the case of a pdf file) are communicated by the upload form to the add work form through a XML file and AJAX. In the repository of XML files there is no one file associated to your username, so for some reasons the script that determines the file data and writes the XML file is never being activated when you upload a file.
Let's try to narrow down the source of the problem. Could you please run the add work form (you may simulate a new contribution and abort it then, without uploading anything) and check that, when you click the "Upload now" button, the URL of the new tab ends with "Special:Upload?cw_file=score" and your username is shown on the top-right side of the page? The script that writes the XML file only works if "cw_file=score" is present and if you're logged in, so, if one of these conditions is not fulfilled, the XML file is not written and there's no communication from the upload form to the add work form.
Max
Let's try to narrow down the source of the problem. Could you please run the add work form (you may simulate a new contribution and abort it then, without uploading anything) and check that, when you click the "Upload now" button, the URL of the new tab ends with "Special:Upload?cw_file=score" and your username is shown on the top-right side of the page? The script that writes the XML file only works if "cw_file=score" is present and if you're logged in, so, if one of these conditions is not fulfilled, the XML file is not written and there's no communication from the upload form to the add work form.
Max
-
- Posts: 53
- Joined: 13 Jul 2016 10:47
Re: Updates to submission process?
I clicked the "Upload now" button and got the new tab
I updloaded the file and entered nothing in my upload tab. Then I got an error, that i did not enter a URL for a media file. So i used again: right mouse button -> copy link address, then Ctrl-v in the upload tab.
Now it worked, bit the result was:
So, no ScoreInfo. Could it be the first error, or my settings in Firefox, Ubuntu 18.04 (no 3. party cookies, do not create a chronicle)?
My user name was shown.
I updloaded the file and entered nothing in my upload tab. Then I got an error, that i did not enter a URL for a media file. So i used again: right mouse button -> copy link address, then Ctrl-v in the upload tab.
Now it worked, bit the result was:
Code: Select all
{{#Legend:}}
*{{PostedDate|2018-12-26}} {{CPDLno|52716}} [[Media:Nh_Silcher_Alle_Jahre_wieder.pdf|{{pdf}}]]
{{Editor|Nikolaus Hold|2018-12-26}}{{ScoreInfo|Unknown||}}{{Copy|Creative Commons Attribution Non-Commercial}}
:'''Edition notes:''' For {{Vcat|SA}} from [[Kinderlieder in 6 Heften (Friedrich Silcher)]].
Re: Updates to submission process?
This confirms that, for some reasons, the upload form does not communicate with the add work form. It is possible that your specific browser settings interfere with the communication mechanism. Some time ago I struggled with Firefox under Linux due to some strange behavio(u)r, which did not occur at all under Windows. Would it be possible for you to use more permissive settings about cookies under Firefox, or just try using an alternative browser (e.g., Chrome)?
Max
Max
-
- Posts: 53
- Joined: 13 Jul 2016 10:47
Re: Updates to submission process?
For my next test, I used a Windows environment:
Win 10, Version 1803,
Google Chrome Version 71.0.3578.98
I uploaded pdf and mxl files with the "Upload now" buttons and initially entered nothing in the upload tab. This led to the known error: the score file name/URL "" appears not to be a valid score file name/URL.
Then I entered both URLs manually - see hardcopy - and clicked on "Add works data".
The result is basically the same (no ScoreInfo):
Win 10, Version 1803,
Google Chrome Version 71.0.3578.98
I uploaded pdf and mxl files with the "Upload now" buttons and initially entered nothing in the upload tab. This led to the known error: the score file name/URL "" appears not to be a valid score file name/URL.
Then I entered both URLs manually - see hardcopy - and clicked on "Add works data".
The result is basically the same (no ScoreInfo):
Code: Select all
{{#Legend:}}
*{{PostedDate|2018-12-28}} {{CPDLno|52732}} [[Media:Nh_Silcher_Vöglein_im_hohen.pdf|{{pdf}}]] [[Media:Nh_Silcher_Vöglein_im_hohen.mxl|{{XML}}]]
{{Editor|Nikolaus Hold|2018-12-28}}{{ScoreInfo|Unknown||}}{{Copy|Creative Commons Attribution Non-Commercial}}
:'''Edition notes:''' From [[Kinderlieder in 6 Heften (Friedrich Silcher)]], Book I, No. 6. {{MXL}}
Re: Updates to submission process?
The error reported is consistent with a lack of communication between the upload form and the add work form, as no URL is being transferred. I confirm that the XML file that is required for the communication between the two forms is not being written, so we have to understand why this happens.Nikolaus Hold wrote:I uploaded pdf and mxl files with the "Upload now" buttons and initially entered nothing in the upload tab. This led to the known error: the score file name/URL "" appears not to be a valid score file name/URL.
I'm currently on travel. I'll be back beginning January. Then I'll activate a specific logging and try to identify the reason through that log.
Max
-
- Posts: 53
- Joined: 13 Jul 2016 10:47
Re: Updates to submission process?
Let us return to the original issue of Richard Mix and your answer.
I do not mind new links, but the main page should be simplified as a whole because it needs "forever" to load (probably the longer the bigger the CLDP gets).
I do not mind new links, but the main page should be simplified as a whole because it needs "forever" to load (probably the longer the bigger the CLDP gets).
If somebody uploads for another one, he has to fill in the corresponding fields anyway, whether they are filled or not. For them it is not an extra effort, but for the others a relief.editor details aren't yet filled in for logged-in users / This is intentional because many submissions are made by another user and not by the editor.
This is not an alternative for me because I always work without a browser history for privacy reasons.I'd suggest to activate the browser function that remembers the values entered previously in the same field, so that you get the field completed right away by your browser.
I do not understand. When I call the form Add_work from the old German instructions for sending in scores, I am first asked to log in. Where is the difference to the new process in the user identification? If there's any loophole about uploading something without logging in with username and password, that's the first thing to do away with.no user identification feature / the traditional process that had no other ways to identify the contributor
Re: Updates to submission process?
This not clear to me. Are you referring to the add work form, or to the main page of the CPDL website? The time to render the main page was never mentioned in this topic, so I'm a little bit confused. Anyway, I agree that the main page of the CPDL website includes many complex DPL queries, which make the rendering of the page quite slow. If you login to the main page and display its source code, you should be able to identify the relevant page performance report:Nikolaus Hold wrote:the main page should be simplified as a whole because it needs "forever" to load (probably the longer the bigger the CLDP gets).
Code: Select all
NewPP limit report
Cached time: 20190101172426
Cache expiry: 0
Dynamic content: true
CPU time usage: 1.104 seconds
Real time usage: 2.412 seconds
Preprocessor visited node count: 5035/1000000
Preprocessor generated node count: 7174/1000000
Post‐expand include size: 31614/2097152 bytes
Template argument size: 8496/2097152 bytes
Highest expansion depth: 16/40
Expensive parser function count: 9/100
Unstrip recursion depth: 0/20
Unstrip post‐expand size: 483/5000000 bytes
The non-automated add work form was originally (year 2005) a simple php script, not integrated within the CPDL wiki and reachable through a URL visible to anybody. The php script could not receive from the wiki any information, such as the username of the contributor, and bots were able to harvest its URL and to make bogus submissions. As a consequence, we received hundreds (or maybe thousands) of bogus submissions made by bots.Nikolaus Hold wrote:I do not understand. When I call the form Add_work from the old German instructions for sending in scores, I am first asked to log in. Where is the difference to the new process in the user identification?
Some years later, when I started taking care of the wiki infrastructure, I changed the URL of the script, and embedded the new URL inside a wiki page that is only visible to logged-in users. Now bots, not being logged-in, cannot read the URL and make bogus submissions. This is the reason why you are now requested to login: it's just to prevent that bots can read the URL of the non-automated add work form and make bogus submissions. However, the php script is still not integrated within the wiki, so it is not aware of the contributor's username. Only the automated add work form is integrated within the wiki, and it can retrieve any information available in the wiki database, including the username.
Regarding the required modifications to the way email fields are managed, there are pros and cons in my opinion, so let's see what other admins think about it. For me, no problem to modify the form for the automatic process if there is consensus. A possible consequence may be that the consistency between the automated and the non-automated form cannot be maintained, and the non-automated form should be eventually discontinued.
Max
-
- Posts: 53
- Joined: 13 Jul 2016 10:47
Re: Updates to submission process?
Richard Mix wrote: "I wonder if we could have a News link on the main page" and my brain immediately associated, aha, this slow page that takes "forever" to load. I quoted "forever", so hopefully it was clear that it was not meant to be literal. My opinion that the main page is slow, comes mainly from morning breakfast, where I take my tablet, first look at the news and then usually take a look at the CPDL.The time to render the main page was never mentioned in this topic
I owe it to your interesting reference to the performace report that I have more solid comparative figures available. On my laptop (elderly Core i3, but good enough to make scores with), I took a few samples and looked at the real time usage:
This morning (11:47 - 11:53 German time) the Main_Page needed 1.129 / 0.145 = 7.8 times as long as the page Friedrich_Silcher. However, the page Orlando_di_Lasso even needed 7.99 / 1.129 = 7.1 times as long as the Main_Page. In the afternoon, when the www2 server was unusually slow (16:51-16:57), the following values resulted:
- Main_Page / Friedrich_Silcher = 11.913 / 0,783 = 15.2
Main_Page / Help:Contents = 11.913 / 0,102 = 116.8
Orlando_di_Lasso / Main_Page = 17.195 / 11.913 = 1.4
- The main page often has a response time of >1 second,
the remaining pages have a response time of <1 second.
1. I sympathize with the long-time gripes of Richard Mix, but as you have stated, this must be discussed with the other administrators. I have nothing more to say about that.user information, email fields
2. The automatic transfer of information through the file-upload does not work for me. Since it does not seem to occur to anyone else, it is a singular and therefore completely insignificant occurrence. I mainly wanted to know if this was a user error on my part. I understood that's not the case.
So we could finish this forum topic.
Re: Updates to submission process?
As manager of the ChoralWiki and with particular responsibility for the Main Page, I made some changes that will speed up the load time. Mostly, I got rid of some not really necessary DPL queries in Browse ChoralWiki, cut the display in CPDL News down to the current and previous month (instead of the previous 5 months), and I reduced the number of days of Most Recent Scores from 11 to 8 (10 days inclusive to 7 days inclusive).
The performance is indeed better:
If the improvement is significant enough, then we can keep the changes. I can (and will) revert any of the changes if these reductions seem to go too far or don't offer enough improvement on load time for the Main Page.
The performance is indeed better:
Code: Select all
<!--
NewPP limit report
Cached time: 20190103010620
Cache expiry: 0
Dynamic content: true
CPU time usage: 0.661 seconds
Real time usage: 1.406 seconds
Preprocessor visited node count: 2355/1000000
Preprocessor generated node count: 6116/1000000
Post‐expand include size: 26288/2097152 bytes
Template argument size: 3876/2097152 bytes
Highest expansion depth: 16/40
Expensive parser function count: 9/100
Unstrip recursion depth: 0/20
Unstrip post‐expand size: 483/5000000 bytes
-->
<!--
Transclusion expansion time report (%,ms,calls,template)
100.00% 1274.673 1 -total
70.52% 898.923 2 Template:Box
68.38% 871.575 1 ChoralWiki:TopMatter
12.77% 162.806 1 ChoralWiki:LatestNews
12.52% 159.579 2 Template:NewsIn
11.43% 145.756 62 Template:NewsOn
6.11% 77.852 1 ChoralWiki:LatestScores
5.46% 69.603 2 Template:NewWorkCount
5.24% 66.739 1 ChoralWiki:SearchSection
4.43% 56.415 1 Template:Bar
-->
Charles H. Giffen
CPDL Board of Directors Chair
Admin at & Manager of ChoralWiki
CPDL Board of Directors Chair
Admin at & Manager of ChoralWiki
-
- Posts: 183
- Joined: 31 Jan 2010 04:51
Re: Updates to submission process?
Since the forum bulletin boards don't show up on my CPDL watchlist I can be slow in acknowledging the thoughtful replies for which Max can be counted on! Still,
1. There are some very confusing redirects about which I asked here:
http://www.cpdl.org/wiki/index.php/Form_talk:Add_work
2. The form asks for: "Name of editor/contributor (First_name Last_name):
Email of editor/contributor:"
and after some scrolling, the bit I always forget: "Your email address (for notification/questions):"
If the editor & contributor are different there's head scratching over which to put on the first line, whereas if everything's adjacent it could just read:
Name of music's editor:
Email of music's editor, if known:
Name of edition's uploader, if different and if Contributor acknowledgement is desired:
Email of uploader/form-filler:
3. A minor complaint is the 'cut-along-dotted lines' output of Add works: I often notice Claude doing cleanup like the following http://www.cpdl.org/wiki/index.php?titl ... did=942672. Can the output be in a box that allows command-a or "select all" for copying?
1. There are some very confusing redirects about which I asked here:
http://www.cpdl.org/wiki/index.php/Form_talk:Add_work
2. The form asks for: "Name of editor/contributor (First_name Last_name):
Email of editor/contributor:"
and after some scrolling, the bit I always forget: "Your email address (for notification/questions):"
If the editor & contributor are different there's head scratching over which to put on the first line, whereas if everything's adjacent it could just read:
Name of music's editor:
Email of music's editor, if known:
Name of edition's uploader, if different and if Contributor acknowledgement is desired:
Email of uploader/form-filler:
3. A minor complaint is the 'cut-along-dotted lines' output of Add works: I often notice Claude doing cleanup like the following http://www.cpdl.org/wiki/index.php?titl ... did=942672. Can the output be in a box that allows command-a or "select all" for copying?